Novel machine learning models to predict pneumonia events in supratentorial intracerebral hemorrhage populations: An analysis of the Risa-MIS-ICH study

Background: Stroke-associated pneumonia (SAP) contributes to high mortality rates in spontaneous intracerebral hemorrhage (sICH) populations. Accurate prediction and early intervention of SAP are associated with prognosis. None of the previously developed predictive scoring systems are widely accepted. We aimed to derive and validate novel supervised machine learning (ML) models to predict SAP events in supratentorial sICH populations.

Methods: The data of eligible supratentorial sICH individuals were extracted from the Risa-MIS-ICH database and split into training, internal validation, and external validation datasets. The primary outcome was SAP during hospitalization. Univariate and multivariate analyses were used for variable filtering, and logistic regression (LR), Gaussian naïve Bayes (GNB), random forest (RF), K-nearest neighbor (KNN), support vector machine (SVM), extreme gradient boosting (XGB), and ensemble soft voting model (ESVM) were adopted for ML model derivations. The accuracy, sensitivity, specificity, and area under the curve (AUC) were adopted to evaluate the predictive value of each model with internal/cross-/external validations.

Results: A total of 468 individuals with sICH were included in this work. Six independent variables [nasogastric feeding, airway support, unconscious onset, surgery for external ventricular drainage (EVD), larger sICH volume, and intensive care unit (ICU) stay] for SAP were identified and selected for ML prediction model derivations and validations. The internal and cross-validations revealed the superior and robust performance of the GNB model with the highest AUC value (0.861, 95% CI: 0.793–0.930), while the LR model had the highest AUC value (0.867, 95% CI: 0.812–0.923) in external validation. The ESVM method combining the other six methods had moderate but robust abilities in both cross-validation and external validation and achieved an AUC of 0.843 (95% CI: 0.784–0.902) in external validation.

Conclusion: The ML models could effectively predict SAP in sICH populations, and our novel ensemble model demonstrated reliable robust performance outcomes despite the populational and algorithmic differences. This attempt indicated that ML application may benefit in the early identification of SAP.

Introduction

Stroke-associated pneumonia (SAP) is the most common infectious complication in spontaneous intracerebral hemorrhage (sICH) individuals, with an estimated incidence of 15–25% in overall stroke populations (1–3). SAP is usually adversely associated with increased mortality, prolonged hospital stays, and poor prognosis (3–5). The current large phase III clinical trials have not found the benefits of routine antibiotic prevention for general stroke individuals (6, 7). Therefore, the accurate prediction and early intervention of SAP might contribute to improving the prognosis. Thus, a reliable model is needed for predicting and monitoring potential SAP, so that exact prophylactic interventions or therapeutic antibiotics can be tailored promptly.

In recent decades, a few studies have indicated several independent risk factors for SAP, including older age (5, 8–13), male sex (8, 9, 13, 14), severe stroke (4, 5, 8–16), intubation (4, 15), nasogastric feeding or dysphagia (4, 8, 16), and deeper location and larger volume of sICH (4, 11, 15). Some of these variables were included in several predictive scoring systems for SAP risk stratifications, such as the A²DS² and PNA scores in Germany (9, 12), and the AIS/ICH-APS scores in China (10, 11), and the ISAN score in the UK (13). However, most scoring systems are designed for acute ischemic stroke (AIS) populations (9, 10, 12, 13), and none of the SAP prediction scoring systems are widely accepted in routine clinical practice.

At present, prediction models based on machine learning (ML) have been applied to predict the occurrence and prognosis of various diseases, which greatly promoted diagnostic performance and facilitated more responsive health systems (17–19). In clinical applications, ML algorithms are applied for risk stratification and prognosis prediction of disease and guide clinicians to apply corresponding measures timely. Compared to traditional scoring systems, ML models show smarter, more accurate, more timely, and more convenient characteristics (18–21). While there is currently no ML model for SAP forecasting. Thus, we aim to derive and validate novel supervised ML models to predict SAP events in supratentorial sICH populations and expect to develop a superior and automatic tool for clinical practice.

Materials and methods

Study design and participants

The data for this analysis were obtained from the retrospective database of the Risk Stratification and Minimally Invasive Surgery in Acute Intracerebral Hemorrhage Patients (Risa-MIS-ICH) study (Clinical Trials Identifier: NCT03862729, https://www.clinicaltrials.gov), which was a multicenter ambispective cohort study. Two centers were involved in the retrospective cohort for this work, including the First Affiliated Hospital, Fujian Medical University (FAHFMU, Fuzhou, Fujian), and Anxi County Hospital (ACH, Quanzhou, Fujian). The FAHFMU subcohort (from January 2015 to July 2020) was for the variable filtrations and model derivations/validations. The ACH subcohort (from June 2019 to April 2021) was introduced into this work for external validation. The study protocol followed the principles of the Declaration of Helsinki and was approved by the ethics committee of FAHFMU (GN: MRCTA, ECFAH of FUM [2018]082) and documented in each center. No informed consent was required for the retrospective cohort. This work was reported in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement (22).

The inclusion and exclusion criteria of the participants are shown as follows:

Inclusion criteria:

• Diagnosed with spontaneous sICH by computed tomography (CT)/CT angiography (CTA) scan, and the interval time from onset to recorded CT/CTA scan ≤ 48 h;

• Glasgow Coma Scale (GCS) score > 5 and no cerebral herniation at admission;

• Onset age ≥ 18 years.

Exclusion criteria:

• With any intracranial etiology of supratentorial hemorrhage of arteriovenous malformation (AVM), arterial aneurysm, hemorrhagic cerebral tumor stroke, hemorrhagic infarction, coagulation disorders, or any other potential organic lesions indicating nonspontaneous sICH;

• Occurrence of infratentorial hemorrhage;

• Evidence of pregnancy, or pre-stroke life expectancy < 3 months.

Additional criteria for SAP prediction model derivations/validations in this work:

• Interval time from onset to admission ≤ 24 h;

• Hospital stay ≥ 48 h;

• Receiving no mechanical ventilation or ventilation time ≤ 24 h before SAP events;

• Underwent recent pulmonary infectious disease or received any antibiotic therapy ≤ 4 weeks;

• Critical data loss about SAP in the laboratory, imaging, or other important clinical information.

According to the present guideline, the diagnosis of sICH participants required radiologic records and exclusion of other organic lesions causing hemorrhage. Only supratentorial sICH participants were enrolled in the Risa-MIS-ICH study, and the participants with the cerebral herniation or low GCS scores usually indicated poor prognosis, which was excluded from the study scope. The exclusion of the juvenile and the pregnant population is for ethical consideration. Furthermore, for the unbiased diagnosis of SAP and the precise analysis, the strict additional criteria had to exclude short-term hospitalization, infection associated with mechanical ventilation, and undefined participants.

The screening process of this work is presented in Figure 1A.

FIGURE 1

Figure 1. Flowchart of the current work. (A) Participant enrollment in the retrospective cohort of the Risa-MIS-ICH study; (B) Data flow from the FAHFMU subcohort; (C) The prediction model derivations and internal/cross-/external validations for SAP events. sICH, supratentorial intracerebral hemorrhage; ML, machine learning; LASSO, least absolute shrinkage and selection operator; SAP, stroke-associated pneumonia.

Variable extractions and primary outcomes

Relevant information about participants were retrieved from the electronic medical record (EMR) systems from each neurological research center. The electronic data capture (EDC, http://61.154.9.209:8090/, RealData Corporation, Ningbo, Zhejiang, China) system was employed for database establishment and data collection. The trained professional clinical research coordinators (CRCs) were commissioned for data entry and follow-up. The Risa-MIS-ICH database included 665 variables and involved information on demographics, pre-stroke comorbidities, onset details, imaging features, laboratory results, complications during hospitalizations, interventions, discharge status, and follow-up information. The collation of the database was performed by professional statisticians, and data analysis was carried out after passing the third-party quality control.

The primary outcome of the current analysis was the occurrence of SAP events during hospitalization, and SAP was defined as a pneumonia not incubating during hospital admission and occurring ≥ 48 h after admission in acute stroke populations. Referring to the diagnostic criteria for hospital-acquired pneumonia (HAP), the diagnostic criteria for SAP were as follows (23, 24): the presence of a new or progressive infiltrate in a chest X-ray or CT scan, plus at least two of the following clinical manifestations: (1) fever (T > 38°C) or hypothermia (T < 36°C), (2) leukocytosis [white blood cell (WBC) count > 10 × 10⁹/L] or leukopenia (WBC count < 4 × 10⁹/L), and (3) nursing-recorded purulent airway secretion. Ventilator-acquired pneumonia (VAP), defined as a pneumonia event after ventilation time > 24 h, was excluded from this work.

Statistical analysis and variable filtration

All statistical analyses were performed using the SPSS software (version 22.0, IBM Corporation, Armonk, NY, USA) and Python (version 3.8.3, Anaconda Distribution, Austin, TX, USA). The current work mainly used the development environment of Jupyter Notebook (version 6.0.3) and invoked the key packaged libraries of NumPy (version 1.18.5), Pandas (version 1.1.5), Scikit-learn (version 0.24.2), SciPy (version 1.5.0), Matplotlib (version 3.4.3), and Lifelines (version 0.26.4). The continuous variables and categorical variables are presented as the mean and standard deviation (SD) or median and interquartile range (IQR) and quantities and percentages.

The screening of variables was performed in the FAHFMU subcohort. As shown in Figure 1B, the study variables were initially screened by univariate analyses. The independent sample Student's t-test was used for normally distributed data, the Mann-Whitney U test was used for nonnormally distributed data, and the chi-square test or Fisher's exact test was used for categorical data. All tests in this work were two-sided, and P < 0.05 was considered statistically significant. To prevent overfitting, the least absolute shrinkage and selection operator (LASSO) regression was used in multivariate analysis and further performed after univariate analyses. Each continuous variable was standardized before performing LASSO regression to improve generalizability. LASSO regression selects the optimal penalty value via the internally installed k-fold cross-validation module (k = 3) and recursively removes the least important variables by vanishing coefficients. Through the above steps, the independent significant variables had nonzero coefficients in LASSO regression and were selected as candidate variables for ML model derivations.

Survival analysis was additionally performed in this work, in which all-cause death after stroke onset was defined as the observed indicator. The survival time was defined as the time interval from stroke onset to all-cause death or follow-up. The survival curves were plotted using the Kaplan–Meier method, and survival rates were compared using the log-rank test.

Model derivations and validations

The flow diagram of the model derivations and validations is presented in Figure 1C. The FAHFMU subcohort was randomly split into the training and validation datasets (7:3), which were used for the model derivations and the internal validation, respectively. The model derivations were performed on the candidate variables by six common basic ML algorithms and one additional ensemble model, of which these six well-established algorithms represented various ML frames and are widely accepted at present (19). The ML models were invoked with mature Python packages, including logistic regression (LR), Gaussian naïve Bayes (GNB), random forest (RF), K-nearest neighbor (KNN), support vector machine (SVM), extreme gradient boosting (XGB), and ensemble soft voting model (ESVM). None of these models was uncertain about demonstrating the optimal performance beforehand. In the training process, six basic ML algorithms were independently fitted with the candidate variables and virtual SAP classifications from the training dataset, and model hyperparameters were optimized with the grid-search algorithm to promote model performance. In detail, the grid-search algorithms tune optimal parameters by internally evaluating model performance repeatedly via the nested k-fold cross-validation module (k = 3 in this work). Before the above steps, ML prediction models with different characteristics were generated, and these processes were termed supervised ML. To improve the robustness of ML models, the additional ESVM was derived by incorporating the aforementioned six algorithms. The ESVM is simply a voting system on the weighted classified outputs of the six basic algorithms, and these processes were termed soft voting.

After model derivations, the validation dataset was automatically inputted into the seven models to obtain the predicted classifications in the internal validation. Receiver operating characteristic (ROC) curves were plotted, and the metrics of accuracy, sensitivity, specificity, and area under the curve (AUC) along with 95% CIs were calculated to evaluate the disease discrimination ability of each model. Further supplementary internal evaluation with advanced robustness was performed with n-repeated k-fold cross-validation (n = 3 and k = 5 in this work). This method repartitions the FAHFMU subcohort into k nonoverlapping folds, where the k-1 folds are used for the model derivations and the other fold is used for validation. After n repetitions, n × k combinations are finally generated for robust validation (25).

Furthermore, this work also introduced the external subcohort, which was not involved in variable filtrations and model derivations. In this process, the entire FAHFMU subcohort was considered the training dataset to retrain the prediction models, and the external subcohort was introduced as the exclusive validation dataset. The technical avenue of training and evaluating the models remained the same as above.

Results

Participants and characteristics

From January 2015 to April 2021, a total of 909 participants were included in the retrospective cohort of the Risa-MIS-ICH study, and 441 of these individuals were excluded due to ventilation > 24 h, ineligible time window, or incomplete data. Finally, 468 individuals (n_FAHFMU = 324, n_ACH = 144) were included in this work. The overall average age was 60.44 (±12.51) years, and 308 (65.8%) of the individuals were male sex. SAP events during hospitalizations occurred in 135 (28.8%) [n_FAHFMU = 97 (29.9%), n_ACH = 38 (26.4%)] individuals. The demographic characteristics, clinical manifestations, imaging features, laboratory tests, and prognostic indicators in the FAHFMU and external subcohorts are summarized in Tables 1, 2, respectively. Differences in the analyzed variables between the two centers are shown in Supplementary Table 1.

TABLE 1

Table 1. Baseline characteristics.

TABLE 2

Table 2. Variables of laboratory results, imaging features, and early clinical interventions.

Variable filtration and importance

According to previous literature and clinical experience (3–5, 8–16), 70 variables related to the study were retained for subsequent analyses. Twenty-five variables were identified as potential predictive factors for SAP by univariate analysis and further LASSO regression was performed (Tables 1, 2). LASSO regression showed that nasogastric feeding (coefficient = 0.14687), airway support (coefficient = 0.09609), unconscious onset (coefficient = 0.05304), surgery for external ventricular drainage (EVD, coefficient = 0.01923), larger sICH volume (estimated with the ABC/2 formula in imaging, coefficient = 0.00625), and intensive care unit (ICU) stay (coefficient = 0.00586) were considered independent influencing factors of SAP (Figures 2, 3).

FIGURE 2

Figure 2. Importance ranking of six independent variables selected by LASSO regression: (1) nasogastric feeding, (2) airway support, (3) unconscious onset, (4) surgery for EVD, (5) larger sICH volume, and (6) ICU stay. EVD, external ventricular drainage; ICU, intensive care unit.

FIGURE 3

Figure 3. Multivariate analysis and variable filtrations with LASSO regression. The tuning parameter (λ) was selected for the minimized MSE in the LASSO model using 10-fold cross-validation. Features with nonzero coefficients were selected while the previous λ value was applied. (A) The MSE was plotted vs. log λ. An optimal λ value of 0.02477 was chosen via the minimum criteria and presented as a black vertical dashed line. (B) LASSO coefficient profiles of the features. Each colored line represents the coefficient of each feature, and six of them were selected as independent variables when λ equals 0.02477. MSE, mean-square error.

Model performance

The ROC curves of the seven models built on the internal validation set were shown in Figure 4A. Among the seven models, GNB demonstrated the optimal efficiency to predict SAP with the highest AUC value (0.861, 95% CI: 0.793–0.930), while the ESVM presented the highest accuracy (0.837, 95% CI: 0.764–0.910) and specificity (0.917, 95% CI: 0.862–0.971). The XGB was the most sensitive, with the highest value (0.692, 95% CI: 0.601–0.784) (Table 3A). The decision curve analyses were performed on both training and validation datasets with seven models, as shown in Supplementary Figure 1. The learning curves presented the evolutions of models with different characteristics and are illustrated in Supplementary Figure 2.

FIGURE 4

Figure 4. ROC curves for SAP on the (A) internal and (B) external validation datasets. A greater AUC value indicated a higher predictive ability of the models. ROC, receiver operating characteristic; AUC, area under the curve.

TABLE 3

Table 3. Performance metrics of the ML models in the FAHFMU validation dataset and external subcohort.

Three repeated five-fold cross-validation were established, and a total of 15 combinations were generated from three splits and five-folds. The AUC values of different models from combinations are summarized and presented as heatmaps in Supplementary Figure 3, and all quantified metrics are listed in Supplementary Table 2. In most random states, the ESVM (frequency = 9/15) and XGB (frequency = 8/15) models remained the optimal models in terms of accuracy and sensitivity, respectively. Unlike the results in internal validation, the LR (frequency = 6/15) and RF (frequency = 10/15) models most often had the highest AUC and specificity values, respectively, with robustness.

External validation

The metrics and ROC curves of each model in external validation are shown in Table 3B and Figure 4B. The LR was superior in AUC value (0.867, 95% CI: 0.812–0.923) in the external validation. While GNB had the highest accuracy (0.833, 95% CI: 0.772–0.894) and sensitivity (0.553, 95% CI: 0.471–0.634), the RF was the most specific (0.962, 95% CI: 0.931–0.993). There was no single algorithm with dominant ability and robustness in the external validation. It is worth mentioning that the ESVM had moderate but robust abilities and achieved AUC, accuracy, sensitivity, and specificity values of 0.843 (95% CI: 0.784–0.902), 0.812 (95% CI: 0.749–0.876), 0.447 (95% CI: 0.366–0.529), and 0.943 (95% CI: 0.906–0.981), respectively, in the external validation. The decision curves for predicting SAP on both FAHFMU and external subcohorts with seven models are illustrated in Supplementary Figure 1.

Outcome and survival analysis

In both the FAHFMU and external subcohorts, participants with SAP suffered from significantly higher hospital costs and prolonged hospital stays (both P < 0.001). Three hundred sixty-four (77.8%) of all eligible 468 participants were followed for survival, and 83 (25.3%) of them had experienced SAP during hospitalization. The mean survival times of participants in the two groups were 44.95 ± 2.78 (95% CI: 39.50–50.40) and 55.77 ± 1.26 (95% CI: 53.30–58.25) months, respectively. The median survival times were not available because mortality was < 50%. The 3-month and 1-year survival rates after onset were 86.9 and 78.3% in SAP participants and 96.7 and 94.2% in non-SAP participants. The Kaplan-Meier curves are plotted in Figure 5. There was a significant difference in survival times between the two groups (log-rank χ² = 20.34, P < 0.001).

FIGURE 5

Figure 5. Kaplan–Meier curves of participants with/without SAP over 1-year follow-up. The colored area represents the 95% confidence intervals of the survival rates.

Discussion

It is critical to identify individuals at high risk for SAP and to further tailor timely prophylactic interventions or therapeutic antibiotics. However, for now, the early prediction of SAP in sICH populations is challenging due to the lack of widely accepted prediction tools, which are important for modern precision medicine and evidence-based medicine (EBM) in this field. Thus, we aimed to derive more effective and automatic sICH-SAP prediction tools in this work. The novel ML prediction models were derived and validated as an attempt to combine artificial intelligence (AI) medical engineering and clinical practice in this field. The major findings were as follows. (1) The incidence rate of sICH-SAP was close to 30%, and the sICH-SAP events significantly contributed to prolonged hospital stays, increased hospital costs, and higher mortality. (2) Six independent predictors for sICH-SAP were identified—nasogastric feeding, airway support, unconscious onset, surgery for EVD, larger sICH volume, and ICU stay. (3) ML prediction models were successfully derived and showed good disease discrimination ability. (4) There was no certain single algorithm with the dominant ability and robustness in cross- and external validations, while the ESVM was considered averaged in metrics and better in robustness in different populations after multiple validations.

Various predictors for SAP were identified in prior literature (4, 5, 8–16). This work screened for independent variables for sICH-SAP events by using univariate and multivariate analyses in the FAHFMU subcohort. Nasogastric feeding, airway support, and unconscious onset were identified as strongly associated risk predictors, which overlapped with the results of previous studies (4, 8–16). Nasogastric feeding and airway support measurement were recognized as SAP predictors, which might bring about secretion disturbances in nasal/oral/tracheal cavities, decreased air filtrations, and even aspiration events (4, 8, 15, 16). These early interventions were secondary to the manifestation of unconsciousness. Previous studies mainly included the ranked variable of the GCS score and rarely adopted the onset manifestations (4, 10, 11, 14–16). In this work, the admission GCS score and unconscious onset were simultaneously introduced into the analyses, and the categorical variable of unconscious onset was independently significant for sICH-SAP. The predictors of larger sICH volume and ICU stay were also reported in previous studies (4, 11, 15) and contributed the least to predicting SAP in this work. The larger sICH volume is a primary factor influencing stroke severity, and ICU stay was a comprehensive intervention secondary to stroke severity and resulted in infectious environments. These aforementioned predictors are usually uncontrollable for actively preventing SAP in clinical practice. However, there were still novel findings in the subgroup analysis that only the surgery for EVD was a significant independent predictor (P < 0.001 in FAHFMU/P = 0.001 in external subcohorts) of all surgical approaches in this work, while EVD was only previously reported as a univariate factor for overall infections (4). On the other hand, the surgery for sICH catheter evacuation did not significantly contribute to SAP events in any univariate analyses (both P = 0.089 in FAHFMU/external subcohorts), which was in accordance with the undifferentiated non-neurologic infections in the MISTIE III trial (26). This suggests that we should continuously focus on the stratification of surgical approaches in the prospective cohort of the Risa-MIS-ICH study for convincing evidence.

To date, apart from the ICH-APS score, none of the SAP prediction models is widely available in clinical practice (8–13). The validation dataset for the ICH-APS score was obtained from the Chinese National Stroke Registry (CNSR) with an AUC value of 0.74 (95% CI: 0.72–0.75). Both our optimal ML prediction models [internal validation: 0.861 (95% CI: 0.793–0.930); 0.867 (95% CI: 0.812–0.923)] and robust ESVM classifiers [internal validation: 0.830 (95% CI: 0.756–0.904); external validation: 0.843 (95% CI: 0.784–0.902)] achieved higher AUC values, indicating greater predictive ability.

Li et al. (26) developed ML models to predict SAP events in Chinese AIS populations, which presented better performance with the highest AUC value of 0.843 (95% CI: 0.803–0.882) than other AIS-SAP prediction scores (0.835 for A²DS², 0.786 for PNA, 0.785 for AIS-APS, and 0.78 for ISAN scores). According to metrics from the literature and this work (27–32), the ML prediction models for SAP showed better performance metrics than traditional scoring systems in both sICH and AIS populations. However, due to incomplete variable collections, horizontal comparisons of different prediction models on the same validation dataset were not possible. Despite the defects, the prediction models usually performed better in internal validation than in external validation due to the intrinsic consistency of original datasets and populational heterogeneity, and the comparisons on their respective original validation datasets usually explained the significance (33).

The published research mainly focused on the mutually separated algorithms. Notably, only the optimal algorithm was mentioned in those articles, although ensemble ML models were reported as successful classifiers with greater performance outcomes in the literature (31, 32). The six basic algorithms used in this work have different characteristics as SAP predictors. RF and LR could identify non-SAP participants better, while XGB could identify SAP participants better. We noted that the predictive ability of one single algorithm was uncertain due to the inconsistent ML algorithmic performance outcomes among the internal/cross-/external validations, and the indeterminacy probably restricted the aforehand model selection and implementation in clinical practice. Therefore, a general and robust model is required for stable predictive ability. Based on the principle of soft voting, we additionally derived ESVM classifier incorporating six basic ML algorithms, which was moderate but surprisingly robust in each metric. Notwithstanding that the occupied machine sources of the ESVM equals the summation of the six basic algorithms, this disadvantage could be ignored by timed training and then pro re nata invoking.

Our current work explored the ML for SAP prediction in sICH individuals. During hospitalization, the clinicians could collect the predictive variables and input these variables into an ML model for a predictive suggestion, so that appropriate precautions and interventions would be timely tailored. While the present ML models are semi-automatic and required manual variable input for now. In the coming decades, the internally installed sophisticated algorithms in the EMR system would ceaselessly learn and then calculate the prediction for high-risk individuals in the prospect via dynamically evaluating the keyed-in clinical manifestations from clinicians, the resulting values from the laboratory information system (LIS), and the captured data from the picture archiving and communication system (PACS) (29, 34). The ML application may greatly improve the work efficiency of clinicians and the accuracy of judgment results.

We have strengths that deserve comments. An external subcohort and multiple forms of validation were introduced in this work. Therefore, there was populational and algorithmic robustness of convincing results. Based on the aforementioned circumstances, we derived novel ensemble models for generalizability, which showed moderate but robust predictive abilities in different populations and were fit for real-world practice. However, there are limitations that should be acknowledged in this work. First, the observational retrospective design might introduce unmanageable bias. Uncontrollable baseline characteristics in the observational study might confound SAP risks and further model derivations/validations. Second, some important variables were missing due to the retrospective collection of data in this work. The National Institute of Health Stroke Scale (NIHSS) score, uniform CT scan parameters, scanning timing, and other unrecorded details were unreachable in the retrospective cohort of the Risa-MIS-ICH study and resulted in the inability to perform horizontal comparisons with external models in this work. Third, not all variables were balanced across the centers, which may bias the results. Although we obtained consistent results based on these imbalanced variables, the influence of the heterogeneity still should not be underestimated. Fourth, there are defects in the deep analyses for SAP. The subgroup analyses on pneumonia severity, radiological features, or pathogenic agents were all absent. A simple overall SAP analysis might be rather rough for complex and heterogenetic pulmonary infections. Future studies on our prospective cohort should continue to resolve the aforementioned problems.

Conclusions

In this work, the authors derived SAP prediction models with ML algorithms in supratentorial sICH populations from multiple centers and performed multiple validations for effective and robust confirmations. The ensemble model was a novel application in this work and showed robust performance outcomes in different populations. Our attempt indicated that ML application may benefit in the early identification of SAP.

Data availability statement

The datasets used and/or analyzed during the current work are available from the corresponding author on reasonable request.

Ethics statement

The studies involving human participants were reviewed and approved by Ethics Committee of First Affiliated Hospital, Fujian Medical University. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

F-XL, D-ZK, Y-XL, and W-HF designed this study. D-LW and J-CC enrolled participants and supervised in data collecting. YZ, W-HF, and L-YZ performed the machine learning and analyzed the data. Z-YG, R-LC, M-PZ, and Z-FX collected the data for the study. KM performed database management and data cleaning. YZ and Y-XL wrote the initial draft. QH, WH, L-YZ, and F-XL reviewed and edited the paper. D-ZK obtained the funding. All authors read and approved the final manuscript.

Funding

This work was funded by the Stroke Prevention and Treatment Project of the National Health Commission—Research and Popularization of Appropriate Intervention Technology for the Stroke High Risk Group in China (GN-2018R002), the Fujian Science and Technology Innovation Joint Fund Project (2019Y9118), and the Technology Platform Construction Project of Fujian Province (2020Y2003 and 2021Y2001).

Acknowledgments

The authors thank all of the individuals who kindly participated in this study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2022.955271/full#supplementary-material

References

1. Kumar S, Selim MH, Caplan LR. Medical complications after stroke. Lancet Neurol. (2010) 9:105–18. doi: 10.1016/S1474-4422(09)70266-2

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Murthy SB, Moradiya Y, Shah J, Merkler AE, Mangat HS, Iadacola C, et al. Nosocomial infections and outcomes after intracerebral hemorrhage: a population-based study. Neurocrit Care. (2016) 25:178–84. doi: 10.1007/s12028-016-0282-6

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Lord AS, Lewis A, Czeisler B, Ishida K, Torres J, Kamel H, et al. Majority of 30-day readmissions after intracerebral hemorrhage are related to infections. Stroke. (2016) 47:1768–71. doi: 10.1161/STROKEAHA.116.013229

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Lord AS, Langefeld CD, Sekar P, Moomaw CJ, Badjatia N, Vashkevich A, et al. Infection after intracerebral hemorrhage: risk factors and association with outcomes in the ethnic/racial variations of intracerebral haemorrhage study. Stroke. (2014) 45:3535–42. doi: 10.1161/STROKEAHA.114.006435

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Tinker RJ, Smith CJ, Heal C, Bettencourt-Silva JH, Metcalf AK, Potter JF, et al. Predictors of mortality and disability in stroke-associated pneumonia. Acta Neurol Belg. (2021) 121:379–85. doi: 10.1007/s13760-019-01148-w

PubMed Abstract | CrossRef Full Text | Google Scholar

6. van de Beek D, Wijdicks EF, Vermeij FH, de Haan RJ, Prins JM, Spanjaard L, et al. Preventive antibiotics for infections in acute stroke: a systematic review and meta-analysis. Arch Neurol. (2009) 66:1076–81. doi: 10.1001/archneurol.2009.176

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Westendorp WF, Vermeij JD, Zock E, Hooijenga IJ, Kruyt ND, Bosboom HJ, et al. The Preventive Antibiotics in Stroke Study (PASS): a pragmatic randomised open-label masked endpoint clinical trial. Lancet. (2015) 385:1519–26. doi: 10.1016/S0140-6736(14)62456-9

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Kwon HM, Jeong SW, Lee SH, Yoon BW. The pneumonia score: a simple grading scale for prediction of pneumonia after acute stroke. Am J Infect Control. (2006) 34:64–8. doi: 10.1016/j.ajic.2005.06.011

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Hoffmann S, Malzahn U, Harms H, Koennecke HC, Berger K, Kalic M, et al. Development of a clinical score (A2DS2) to predict pneumonia in acute ischemic stroke. Stroke. (2012) 43:2617–23. doi: 10.1161/STROKEAHA.112.653055

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Ji R, Shen H, Pan Y, Wang P, Liu G, Wang Y, et al. Novel risk score to predict pneumonia after acute ischemic stroke. Stroke. (2013) 44:1303–9. doi: 10.1161/STROKEAHA.111.000598

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Ji R, Shen H, Pan Y, Du W, Wang P, Liu G, et al. Risk score to predict hospital-acquired pneumonia after spontaneous intracerebral haemorrhage. Stroke. (2014) 45:2620–8. doi: 10.1161/STROKEAHA.114.005023

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Friedant AJ, Gouse BM, Boehme AK, Siegler JE, Albright KC, Monlezun DJ, et al. A simple prediction score for developing a hospital-acquired infection after acute ischemic stroke. J Stroke Cerebrovasc Dis. (2015) 24:680–6. doi: 10.1016/j.jstrokecerebrovasdis.2014.11.014

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Smith CJ, Bray BD, Hoffman A, Meisel A, Heuschmann PU, Wolfe CD, et al. Can a novel clinical risk score improve pneumonia prediction in acute stroke care? A UK multicenter cohort study. J Am Heart Assoc. (2015) 4:e001307. doi: 10.1161/JAHA.114.001307

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Marini S, Morotti A, Lena UK, Goldstein JN, Greenberg SM, Rosand J, et al. Men experience higher risk of pneumonia and death after intracerebral hemorrhage. Neurocrit Care. (2018) 28:77–82. doi: 10.1007/s12028-017-0431-6

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Divani AA, Hevesi M, Pulivarthi S, Luo X, Souslian F, Suarez JI, et al. Predictors of nosocomial pneumonia in intracerebral hemorrhage patients: a multi-center observational study. Neurocrit Care. (2015) 22:234–42. doi: 10.1007/s12028-014-0065-x

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Lioutas VA, Marchina S, Caplan LR, Selim M, Tarsia J, Catanese L, et al. Endotracheal intubation and in-hospital mortality after intracerebral hemorrhage. Cerebrovasc Dis. (2018) 45:270–8. doi: 10.1159/000489273

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Gao Y, Cai GY, Fang W, Li HY, Wang SY, Chen L, et al. Machine learning based early warning system enables accurate mortality risk prediction for COVID-19. Nat Commun. (2020) 11:5033. doi: 10.1038/s41467-020-18684-2

PubMed Abstract | CrossRef Full Text

18. Chen CY, Lin WC, Yang HY. Diagnosis of ventilator-associated pneumonia using electronic nose sensor array signals: solutions to improve the application of machine learning in respiratory research. Respir Res. (2020) 21:45. doi: 10.1186/s12931-020-1285-6

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Deo RC. Machine learning in medicine. Circulation. (2015) 132:1920–30. doi: 10.1161/CIRCULATIONAHA.115.001593

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Handelman GS, Kok HK, Chandra RV, Razavi AH, Lee MJ, Asadi H. eDoctor: machine learning and the future of medicine. J Intern Med. (2018) 284:603–19. doi: 10.1111/joim.12822

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Raju B, Jumah F, Ashraf O, Narayan V, Gupta G, Sun H, et al. Big data, machine learning, and artificial intelligence: a field guide for neurosurgeons. J Neurosurg. (2020) 2:1–11. doi: 10.3171/2020.5.JNS201288

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. (2015) 350:g7594. doi: 10.1136/bmj.g7594

PubMed Abstract | CrossRef Full Text | Google Scholar

23. American Thoracic Society, Infectious Diseases Society of America. Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. Am J Respir Crit Care Med. (2005) 171:388–416. doi: 10.1164/rccm.200405-644ST

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Kalil AC, Metersky ML, Klompas M, Muscedere J, Sweeney DA, Palmer LB, et al. Management of adults with hospital-acquired and ventilator-associated pneumonia: 2016 clinical practice guidelines by the infectious diseases society of America and the American Thoracic Society. Clin Infect Dis. (2016) 63:e61–111. doi: 10.1093/cid/ciw353

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Parvandeh S, Yeh HW, Paulus MP, McKinney BA. Consensus features nested cross-validation. Bioinformatics. (2020) 36:3093–8. doi: 10.1093/bioinformatics/btaa046

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Hanley DF, Thompson RE, Rosenblum M, Yenokyan G, Lane K, McBee N, et al. Efficacy and safety of minimally invasive surgery with thrombolysis in intracerebral haemorrhage evacuation (MISTIE III): a randomised, controlled, open-label, blinded endpoint phase 3 trial. Lancet. (2019) 393:1021–32. doi: 10.1016/S0140-6736(19)30195-3

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Li X, Wu M, Sun C, Zhao Z, Wang F, Zheng X, et al. Using machine learning to predict stroke-associated pneumonia in Chinese acute ischaemic stroke patients. Eur J Neurol. (2020) 27:1656–63. doi: 10.1111/ene.14295

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Abedi V, Avula V, Chaudhary D, Shahjouei S, Khan A, Griessenauer CJ, et al. Prediction of long-term stroke recurrence using machine learning models. J Clin Med. (2021) 10:1286. doi: 10.3390/jcm10061286

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Sadasivuni S, Saha M, Bhatia N, Banerjee I, Sanyal A. Fusion of fully integrated analog machine learning classifier with electronic medical records for real-time prediction of sepsis onset. Sci Rep. (2022) 12:5711. doi: 10.1038/s41598-022-09712-w

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Mainali S, Darsie ME, Smetana KS. Machine learning in action: stroke diagnosis and outcome prediction. Front Neurol. (2021) 12:734345. doi: 10.3389/fneur.2021.734345

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Muhlestein WE, Akagi DS, McManus AR, Chambless LB. Machine learning ensemble models predict total charges and drivers of cost for transsphenoidal surgery for pituitary tumor. J Neurosurg. (2018) 131:507–16. doi: 10.3171/2018.4.JNS18306

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Shah AA, Devana SK, Lee C, Bugarin A, Lord EL, Shamie AN, et al. Machine learning-driven identification of novel patient factors for prediction of major complications after posterior cervical spinal fusion. Eur Spine J. (2022) 31:1952–9. doi: 10.1007/s00586-021-06961-7

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Hotter B, Hoffmann S, Ulm L, Meisel C, Bustamante A, Montaner J, et al. External validation of five scores to predict stroke-associated pneumonia and the role of selected blood biomarkers. Stroke. (2021) 52:325–30. doi: 10.1161/STROKEAHA.120.031884

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Rana S, Luo W, Tran T, Venkatesh S, Talman P, Phan T, et al. Application of machine learning techniques to identify data reliability and factors affecting outcome after stroke using electronic administrative records. Front Neurol. (2021) 12:670379. doi: 10.3389/fneur.2021.670379

PubMed Abstract | CrossRef Full Text | Google Scholar