A comparative study of machine learning algorithms for predicting acute kidney injury after liver cancer resection

Introduction

Acute kidney injury (AKI) is a common postoperative complication among surgical patients. The incidence of postoperative AKI accounts for 18%–47% of total hospitalized AKI patients (Tang & Murray, 2004). Postoperative AKI can prolong the hospitalization period and increase the risk of both in-hospital mortality and chronic kidney disease. Clinically, postoperative AKI is easy to overlook, and the diagnostic rate is low (Moore et al., 2010; Bennet et al., 2010).

Hepatectomy is the most common aggressive treatment for primary liver cancer. In order to control hemorrhaging during surgery, it is often necessary to block the hepatic portal. This can disturb liver microcirculation. Hepatic ischemia-reperfusion injury occurs after the hepatic portal is opened, releasing a large amount of inflammatory media and oxygen free radicals, thus inhibiting liver function. At the same time, due to surgical trauma, decreased circulation in the liver and kidneys, the release of granulocyte elastase and other factors, postoperative renal damage is also common (Miranda et al., 2010). Therefore, although progress has been made on hepatectomy, the occurrence of AKI remains an important factor influencing prognosis (Nadeem et al., 2014).

Many studies have used classical regression methods to identify risk factors and construct risk prediction models. However, a non-linear relationship between explanatory variables and outcome variables cannot be ruled out (Chen et al., 2011; Vives et al., 2016; Jun et al., 2018). However, compared with conventional analysis methods, machine learning techniques minimize these limitations and may perform better. Studies have shown that machine learning can predict AKI after liver transplant, cardiac surgery, severe burns and percutaneous coronary intervention (Lee et al., 2018a; Lee et al., 2018b; Tran et al., 2019; Huang et al., 2018). Other studies have shown that decision tree algorithms can predict hospitalized patients’ AKI risk after surgery (Thottakkara et al., 2016). Studies have also shown that support vector machines can be used as risk prediction models for postoperative AKI in septic patients (Mohamadlou et al., 2018) . This study investigated the preoperative risk factors associated with secondary AKI after hepatectomy. It used machine learning techniques (logistic regression, decision tree and GradientBoosting) to construct a predictive model of secondary AKI after hepatectomy, thus providing guidance for clinical therapies, and improving surgical patient prognosis.

Materials and Methods

Results

The pandas_profiling package was applied to data exploration (see attachment Appendix S1 for the results) with Python. The analysis included 1,173 hepatectomy patients, including 77 patients (6.6%) with AKI and 1,096 (93.4%) without. The BMI values of the two groups were different, and the difference was statistically significant (P < 0.040). Neither age nor tumor size showed statistically significant difference between the two groups (see Table 1).

Figure 1 demonstrates that age, tumor size and surgery duration have weak positive correlations with AKI. Cholesterol and PLT each had weak negative correlations with AKI.The Gbdt algorithm model importance matrix is shown in Fig. 2. Age, cholesterol, tumor size, surgery duration and PLT are the five most influential factors.

In Table 2 and Fig. 3, the models constructed by the four machine learning algorithms in the training group are compared. Among the four machine learning algorithms, random forest and gbm have the highest accuracy, 0.989 and 0.970 respectively. The precision of four of the five algorithms is 1, with random forest as the lone exception. The highest recall rate was that of the random forest algorithm (0.852). Among the four algorithms, random forest had the highest recall rate and f1 score, 0.852 and 0.911, respectively. The AUC values for the four algorithms were: gbm (0.999), forest (0.997), Gbdt (0.963) and DecisionTree (0.806). Among the four algorithms, random forest had the lowest MSE value (0.011).

In Table 3 and Fig. 4, the models constructed by four machine learning algorithms in the test group are compared. Gbm had the highest accuracy (0.932). Random forest and gbm had the highest precision, both being 0.333. The recall rate for the random forest algorithm was 0.087. The lowest f1 score was that of decision tree at 0.059. The AUC values of the four algorithms were: Gbdt (0.772), gbm (0.725), forest (0.662) and DecisionTree (0.628). Among the four algorithms, gbm had the lowest MSE value at 0.068.

Discussion

Hepatectomy is an effective therapy for primary liver cancer. To block interoperative bleeding, it is often necessary to block the hepatic hilum. This can induce hepatic ischemia-reperfusion injury. It can cause not only liver dysfunction, but also kidney injury (Sheridan et al., 2016; Gao et al., 2018). At the same time, due to surgical trauma, decreased blood flow in the liver and decreased kidney circulation, granulocyte elastase release and other factors, postoperative renal damage can also occur (Fonseca-NetoI et al., 2012). In this study, machine learning techniques compared the predictive accuracy of AKI predictions after hepatectomy. The Gbdt algorithm indicated that age, cholesterol, tumor size, surgery duration and PLT were the five most important weights for AKI. The results showed that Gbdt had the highest AUC in both training and test groups. Thus, it could predict the likelihood of AKI. All four machine learning algorithms could predict the likelihood of AKI as well. The accuracy was greater than 90%, and the MSE values were less than 0.1.

Studies (Craig et al., 2001; Yim et al., 2000; Amar et al., 2007) have shown that laparoscopic surgery can reduce postoperative inflammatory response indicator levels, including C-reactive protein, interleukin and reactive oxygen species in neutrophils. These inflammatory mediators have been shown to be identical to the inflammatory mediators in AKI (Wu et al., 2014). In addition, triglyceride deposition around the renal tubules can cause high levels of free fatty acids around the kidney cells. This can impair kidney function (Levine et al., 1997). Zhang et al. (2011) analyzed 3,336 patients from 19 related studies covering 11 countries and found that blood CysC is a good predictor of acute kidney injury. It also has high specificity and accuracy for early kidney injury. These findings are similar to those of the present study.

Ongoing studies (Slankamenac et al., 2009) also show that diabetes, high BMI and low postoperative albumin are risk factors for postoperative AKI. Diabetes is a well-known risk factor for various postoperative AKIs, including hepatectomy. Low serum albumin concentrations have recently been associated with various postoperative AKIs (Cho et al., 2014). Moreover, studies (Wu et al., 2020) have also shown that the lowest platelet count over the first 48 h is a new biomarker for AKI. This study’s findings support these views.

The goal of logistic regression in statistics is different from that of logistic regression in machine learning. By default, there is a potential law in statistics. There are various restrictions in adjusting the model to meet the assumption conditions to find the potential law. However, machine learning is different; it is only concerned with the deviation between predicted and real values. Moreover, the integration algorithm adopted in this study considers more information gain when calculating. Thus, it naturally eliminates linear correlation, and also prevents non-linear correlation.

In addition, variables are often screened with principal component analysis (Zhang & Castelló, 2017). However, principal component analysis is not always required in machine learning algorithms. It is used excessively to screen features. Doing so can omit important factors for outcome variables. In the real world, no clinical factor affecting prognosis should be ignored.

This study has several limitations. Firstly, not all confounding factors could be controlled, as this was a retrospective study. Secondly, caution should be exercised in interpreting the study’s results since it was a single-center study in which all surgeries were performed by an experienced surgeon. Thirdly, the machine learning techniques’ performance may vary when applied to larger samples with different covariate distributions. This study only performed internal, and not external, verification. In addition, different parameters in machine learning can result in different AUC values. Corresponding models are needed for different occasions according to needs, and should not excessively prioritize AUC values. Furthermore, an exorbitant AUC value may be unsuitable, as the accuracy, precision and recall rates may fall to unacceptable levels. This would make models unreliable in real world applications when the AUC value is high. Although most of the important reported variables are not clinically modifiable, appropriate measures could be taken to personalize prevention based on AKI risk.

Conclusion

This study shows that all four machine learning techniques can predict AKI likelihood, among which GradientBoosting performs the best. At the same time, the Gbdt algorithm suggests that age, cholesterol, tumor size, surgery duration and PLT are the five most important weights for the likelihood of acute kidney injury after liver cancer resection.

Supplemental Information