Introduction

Acute-on-chronic liver failure (ACLF) is a progressive disease associated with rapid clinical deterioration and high mortality. Artificial liver support system (ALSS) therapy is an available treatment for patients with ACLF and is a bridge to liver transplantation1. However, the optimal extracorporeal anticoagulation regimen for ALSS therapy remains uncertain. Regional citrate anticoagulation (RCA) is now the preferred anticoagulation method for patients underwent continuous renal replacement therapy (CRRT)2,3. RCA seems also safe and feasible for patients with liver failure, and citrate accumulation is well tolerated by them4,5,6,7,8,9,10,11,12,13,14. The blood purification techniques used in these patients include a dialysis technique that can remove citrate directly. Our previous study suggests that RCA is relatively safe and effective in patients with ACLF receiving double plasma molecular adsorption system plus plasma exchange (DPMAS plus PE) therapy that does not include dialysis and filtration techniques15. However, transient citrate accumulation was found in all patients due to PE therapy in both groups and RCA in RCA group, and it remained much higher in RCA group than that in heparin anticoagulation group 2 h after the conclusion of the ALSS therapy (34.0% vs. 7.4%, p = 0.000)15.

Citrate accumulation is a feared and potentially lethal complication of RCA manifesting as an increased ratio of total calcium (Catot) to ionized calcium (Caion) with or without hypocalcaemia, metabolic acidosis and enlarged anion gap8. Early prediction may help avoid or reduce the risk of citrate accumulation. Schultheiß et al. found that standard laboratory liver function parameters showed poor predictive capabilities regarding citrate accumulation and that serum lactate ≥ 3.4 mmol/L and prothrombin time activity (PTA) ≤ 26% predicted an increase of citrate accumulation with high sensitivity (86% for both lactate and PTA) and specificity (86% for lactate, 92% for PTA)8. The results were obtained in critically ill patients with decompensated liver cirrhosis or acute liver failure treated with continuous venovenous hemodialysis (CVVHD), though they may also be useful for patients with ACLF treated with ALSS therapy that includes dialysis and filtration techniques. Our previous study revealed that gender and baseline lactate were independent predictors for citrate accumulation in patients with ACLF who received DPMAS plus PE therapy with heparin anticoagulation. In that study, an increase of citrate accumulation due to PE therapy was predicted by the presence of baseline levels of plasma lactate ≥ 2.65 mmol/L [sensitivity 62.5%, specificity 84.2%, area under the receiver operating curves (AUROCs) = 0.750, 95% confidence interval (CI) = 0.601–0.899]16. However, whether these results could be applied to patients with ACLF who receive ALSS therapy without dialysis and filtration techniques is unclear. In our retrospective study, we developed a model to predict longer duration of citrate accumulation (LDCA) in patients with hepatitis B virus (HBV) infection-related liver injury treated with DPMAS plus PE therapy with RCA.

Results

Patient characteristics

A total of 480 patients treated with ALSS therapy were initially screened and enrolled (Fig. 1). Patients treated with non-DPMAS plus PE therapy (N = 7) or non-RCA (N = 9) were excluded from the study. Patients with liver cancer (N = 18) and those without HBV infection (N = 108) were also excluded. A total of 338 patients were enrolled and randomly divided into a derivation cohort (N = 230) and a validation cohort (N = 108) with a ratio of 2:1 using SPSS software. All patients were followed up 2 h after RCA-ALSS therapy.

Figure 1
figure 1

Flow diagram of patient selection. Of the 480 patients in our database that received ALSS therapy, 142 were excluded from the study. The remaining 338 patients were randomly divided into a derivation cohort (N = 230) and a validation cohort (N = 108). ALSS artificial liver support system, DPMAS double plasma molecular adsorption system, PE plasma exchange, RCA regional citrate anticoagulation, HBV hepatitis B virus.

Full size image

The patient characteristics are shown in Table 1. There were no significant differences between the two cohorts in gender, age, causes of liver disease, usage of antiviral agents, or laboratory parameters before the initial ALSS therapy. The Model for End-Stage Liver Disease (MELD) score17 and the proportion who met the HBV-ACLF criteria18 were similar in the two cohorts. The overall rates of longer duration of citrate accumulation (LDCA) were not significantly different between the two cohorts. There were no significant differences in indicators representing that patients received similar RCA, such as intracorporeal Catot before RCA-ALSS therapy, intracorporeal and extracorporeal Caion during RCA-ALSS therapy, and intracorporeal Catot and Caion 2 h after RCA-ALSS therapy (Table 1).

Table 1 Characteristics of derivation and validation cohorts.
Full size table

Development of R-CA model in derivation cohort

The derivation cohort was analyzed for the predictors of LDCA on the basis of baseline parameters by logistic regression analysis. In multivariate analysis, four baseline variables were found to be independently associated with LDCA: gender, international normalized ratio (INR) of prothrombin time (PT) (PT-INR), serum creatinine, and serum chloride (Table 2). A predictive R-CA model, Logit (P) = 5.380 + 1.173 × Gender + 0.797 × PT-INR + 1.863 × Serum creatinine (× ULN)# − 0.109 × Serum chloride (mmol/L) [gender: female = 2, male = 0; #Statistical analysis using relative values adjusted by gender (a multiple of upper limit of normal)], was developed by using multivariate logistic regression analysis with the backward stepwise (likelihood ratio) method.

Table 2 Predictors for LDCA in derivation cohort.
Full size table

Testing of R-CA model in validation cohort

Before developing a simplified predictive score, the four independent predictors were tested in the validation cohort. Gender, PT-INR, serum creatinine, and serum chloride were verified as independent predictors of LDCA based on the results of the multivariate analysis of the validation cohort (Table 3). The R-CA model showed a good predictability with AUROC of 0.848 (95% CI 0.795–0.892, p = 0.000) and 0.856 (95% CI 0.776–0.916, p = 0.000) in the derivation and validation cohorts, respectively. The expected LDCA rates and observed LDCA rates from the derivation cohort (R2 = 0.909, p = 0.000) matched with the validation cohort (R2 = 0.778, p = 0.007; Fig. 2A).

Table 3 Testing predictors for LDCA in validation cohort.
Full size table
Figure 2
figure 2

Linear correlation lines of expected LDCA rate and observed LDCA rate of R-CA model and R-CA score. (A) The linear correlation lines of expected and observed LDCA rates in the derivation and validation cohorts based on the R-CA model. (B) The linear correlation lines of expected and observed LDCA rates in the derivation and validation cohorts based on the R-CA score. The expected and observed LDCA rates of the derivation cohort match those of the validation cohort. LDCA longer duration of citrate accumulation, R-CA model logistic regression model of risk predictors for citrate accumulation, R-CA score risk score for citrate accumulation.

Full size image

Development of R-CA score

Following the validation of our predictive model, the four individual parameters were scored. An ordinal grading system (0–2) with distinct hazard ratios (HR) on logistic regression was performed by comprehensively considering their cut-off values of AUROCs predicting the probability of LDCA, ACLF diagnostic criteria19,20, and clinically significant values (Tables 4, 5). The total R-CA score ranges from a minimum of 0 to a maximum of 8. The scoring parameters are easy to collect laboratory measurements or clinical features that showed a distinct hazard ratio on logistic regression in derivation and validation cohorts (all HR > 2 and all p = 0.000; Table 6).

Table 4 Simplified univariate predictors for LDCA in derivation and validation cohorts.
Full size table
Table 5 R-CA score.
Full size table
Table 6 Predictive model for LDCA in derivation and validation cohorts, and that among patients with or without liver cirrhosis.
Full size table

The R-CA scores and LDCA rates showed a linear correlation in the derivation cohort (R2 = 0.912, p = 0.000; Fig. 3). A linear regression equation was developed: LDCA rate = 12.8% × R-CA score − 1.2%. The expected LDCA rates and observed LDCA rates based on the R-CA scores in the derivation cohort (R2 = 0.845, p = 0.000) matched those of the validation cohort (R2 = 0.842, p = 0.000; Fig. 2B).

Figure 3
figure 3

Linear regression lines of R-CA score and observed LDCA rate in the derivation cohort. A linear regression equation was developed for the R-CA scores and LDCA in derivation. LDCA longer duration of citrate accumulation, R-CA score risk score for citrate accumulation.

Full size image

Evaluation of R-CA model and R-CA score as predictors of LDCA in derivation and validation cohorts

Our R-CA model and its simplified R-CA score were compared with other potential predictors, such as the MELD score, PT-INR and lactate (Fig. 4A,B and Table 7). AUROCs of the R-CA model and the R-CA score in the derivation cohort were 0.848 and 0.803, and those in the validation cohort were 0.856 and 0.816, respectively. Our R-CA model was found to be as capable as the R-CA score, and superior to the MELD score (AUROC = 0.725) and other univariate predictors (AUROCs < 0.700), in predicting LDCA (p = 0.369, p = 0.022 and p ≤ 0.001, respectively). The R-CA score was as capable as the MELD score in predicting LDCA (p = 0.174) and superior to other univariate predictors in predicting LDCA (p < 0.05).

Figure 4
figure 4

Receiver operating curves (ROC) for the abilities of risk models to predict LDCA. ROC for risk models predicting 3-month mortality in the derivation cohort (A), validation cohort (B), patients with liver cirrhosis (C), and patients without liver cirrhosis (D). Our R-CA model and R-CA score were as capable as or superior to all other models in predicting LDCA. R-CA model logistic regression model of risk predictors for citrate accumulation, R-CA score risk score for citrate accumulation, MELD Model for End-Stage Liver Disease, PT-INR international normalized ratio (INR) of prothrombin time (PT).

Full size image
Table 7 Comparison of the predictive values of R-CA model, R-CA score, and other predictors in derivation and validation cohorts.
Full size table

Less than 10% of patients who had an R-CA model ≤  − 1.00 or an R-CA score of 0–2 experienced LDCA. R-CA model ≤  − 1.00 had an AUROC of 0.848, a sensitivity of 74.1%, a specificity of 88.6%, a positive predictive value of 66.7%, and a negative predictive value of 91.8%. R-CA score of 0–2 had an AUROC of 0.803, a sensitivity of 70.4%, a specificity of 84.1%, a positive predictive value of 57.6%, and a negative predictive value of 90.2% (Table 8). Although patients with MELD score ≤ 26.6, PT-INR ≤ 2.43 and lactate ≤ 2.65 had high negative predictive values of LDCA, their AUROCs were all less than 0.750 (Table 8).

Table 8 Predictive values of predictors based on their maximum area of AUROCs in derivation cohort and patients with liver cirrhosis.
Full size table

Evaluation of R-CA model and R-CA score as predictors of LDCA among patients with or without liver cirrhosis

The R-CA model and R-CA score also showed a distinct hazard ratio on logistic regression among patients with or without liver cirrhosis (all HR > 2 and all p = 0.000; Table 6). The expected LDCA rates and observed LDCA rates based on the R-CA model among patients with liver cirrhosis (R2 = 0.951, p = 0.000) matched with patients without liver cirrhosis (R2 = 0.754, p = 0.001; Fig. 5A). The expected LDCA rates and observed LDCA rates based on the R-CA scores among patients with liver cirrhosis (R2 = 0.799, p = 0.001) also matched those of patients without liver cirrhosis (R2 = 0.640, p = 0.006; Fig. 5B).

Figure 5
figure 5

Linear correlation lines of expected LDCA rate and observed LDCA rate of R-CA model and R-CA score among patients with or without liver cirrhosis. (A) The linear correlation lines of expected and observed LDCA rates among patients with or without liver cirrhosis based on the R-CA model. (B) The linear correlation lines of expected and observed LDCA rates among patients with or without liver cirrhosis based on the R-CA score. The expected and observed LDCA rates of patients with liver cirrhosis match those of patients without liver cirrhosis. LDCA longer duration of citrate accumulation, R-CA model logistic regression model of risk predictors for citrate accumulation, R-CA score risk score for citrate accumulation.

Full size image

AUROCs of the R-CA model and the R-CA score among patients with liver cirrhosis were 0.851 and 0.804, and those among patients without liver cirrhosis were 0.836 and 0.781, respectively (Fig. 4C,D, and Table 9). Our R-CA model was found to be as capable as the R-CA score, and superior to the MELD score (AUROC = 0.712) and other univariate predictors (AUROCs < 0.700), in predicting LDCA (p = 0.280, p = 0.005 and p = 0.000, respectively). The R-CA score was as capable as the MELD score in predicting LDCA (p = 0.075) and superior to other univariate predictors in predicting LDCA (p < 0.01).

Table 9 Comparison of the predictive values of R-CA model, R-CA score, and other predictors among patients with or without liver cirrhosis.
Full size table

About 12.5% of cirrhotic patients who had an R-CA model ≤  − 1.39 or an R-CA score of 0–2 experienced LDCA. R-CA model ≤  − 1.39 had an AUROC of 0.851, a sensitivity of 76.5%, a specificity of 80.5%, a positive predictive value of 61.2%, and a negative predictive value of 89.5%. R-CA score of 0–2 had an AUROC of 0.804, a sensitivity of 70.6%, a specificity of 81.1%, a positive predictive value of 60.0%, and a negative predictive value of 87.3% (Table 8). Although cirrhotic patients with MELD score ≤ 26.6, PT-INR ≤ 2.22 and lactate ≤ 2.65 had high negative predictive values of LDCA, their AUROCs were all less than 0.750 (Table 8).

Correlation between R-CA model, R-CA score, LDCA and disease severity

R-CA model, R-CA score, LDCA, and Catot/Caion 2 h after RCA-ALSS therapy were positively correlated with disease severity rated by MELD score in derivation and validation cohorts and among patients with or without liver cirrhosis with all the p < 0.01 (Fig. 6, Table 10).

Figure 6
figure 6

Correlation between R-CA model, R-CA score, LDCA and disease severity in derivation cohort. R-CA model, R-CA score and LDCA are positively correlated with disease severity rated by MELD score. MELD Model for End-Stage Liver Disease, R-CA model logistic regression model of risk predictors for citrate accumulation, R-CA score risk score for citrate accumulation, LDCA longer duration of citrate accumulation, Catot total calcium, Caion ionized calcium, Catot/Caion Catot to Caion ratio, RCA regional citrate anticoagulation, ALSS therapy artificial liver support system therapy.

Full size image
Table 10 Correlation of R-CA model, R-CA score, LDCA and disease severity rated by MELD score in derivation and validation cohorts, and that among patients with or without liver cirrhosis.
Full size table

Discussion

We developed a predictive model of citrate accumulation for patients with liver failure treated with RCA-ALSS therapy. We found that four variables (gender, PT-INR, serum creatinine, and serum chloride) are independent predictors of LDCA. A predictive R-CA model and its simplified R-CA score have both been developed. Prior to our study, the existing predictors in the literature were all univariate, such as PT-INR and lactate8,16. There was no score dedicated to LDCA in patients with liver failure treated with RCA-ALSS therapy before our study.

Our R-CA model was constructed based on the four most significant variables found in our multivariate analysis. Female gender, serum creatinine, and PT-INR were positively correlated with LDCA, while serum chloride was negatively correlated. It has been reported that the metabolism of citrate is mainly in the mitochondria of liver, muscle, and kidney cells21. This helps to explain the positive correlations between LDCA and gender, serum creatinine, and PT-INR. LDCA may be more likely to occur in female patients because, on average, men have more muscle mass than women, allowing for more citrate metabolism in muscle cells22. As serum creatinine is a marker for renal function, it is expected that both citrate and serum creatinine levels will be high if metabolism in the kidneys is slowed. Additionally, as coagulation failure is a symptom of ACLF, and PT-INR values have been correlated with the degree of liver injury, which is related to the function of liver cells’ mitochondria, a high PT-INR indicates a lower ability of liver cells to metabolize citrate1. The negative correlation between serum chloride and LDCA may be explained by the anion gap. Chloride is one of the most important anions in the body, and its decrease may indicate the increase of other anions, such as lactate, to maintain the anion gap. The correlation coefficient between serum chloride and lactate is − 0.173 (p = 0.001) (Data not shown). Our previous study reveals that baseline Catot/Caion is an independent predictor for citrate accumulation in patients with ACLF who received DPMAS plus PE therapy with heparin anticoagulation (AUROC = 0.725)16. However, the role of the anion gap in predicting LDCA requires further investigation.

We found no correlation between LDCA and standard laboratory liver function parameters, which is in line with the results of previous studies9,12. We predicted that age may be negatively correlated with LDCA, due to decreased muscle mass and weakened mitochondria23,24, but that is not supported by our study.

Our R-CA model is based on some of the same parameters used to determine the MELD score (serum creatinine and PT-INR), which has been used for organ allocation in patients with end-stage liver disease17. We found an AUROC of 0.725 when using the MELD score to predict LDCA, which is inferior to our R-CA model. Our results did not support lactate and PT-INR as predictive factors, though they have been reported in previous studies8,16. Although our R-CA model is able to successfully predict LDCA, complex calculations are required, much like when calculating a MELD score17. To make the model more useful, we developed the R-CA score, a simplified version that does not require complex variables and can be used easily at the patient’s bedside. The R-CA score is constructed based on the four most significant and readily available variables found in our multivariate analysis, and it is as capable as our R-CA model and the MELD score in predicting LDCA. Thus, it has a distinct advantage over our R-CA model and the MELD score. The simple scoring system assigns 0–2 points for each of the four parameters, much like the Child–Turcotte–Pugh (CTP) score for patients with liver cirrhosis and the CLIF-C ACLF score for patients with liver failure25,26. The cutoff values were determined using AUROC, ACLF diagnostic criteria19,20, and clinically significant values. The R-CA score was verified using a validation cohort, and found to be successful in predicting LDCA (AUROC = 0.803 and 0.816 in the derivation and validation cohorts, respectively). The CLIF-C ACLF score, one of the most important but easy-to-use prognostic scores for disease severity in patients with ACLF26, probably also has the capability in predicting LDCA because of some of the same parameters (serum creatinine and PT-INR). We had tried to compare the CLIF-C ACLF score to our score, but the respiratory indicator (arterial blood gas analysis result) was missing in most cases in this retrospectively study. The role of CLIF-C ACLF score in predicting LDCA requires further investigation.

In critically ill patients with acute kidney injury without liver disease treated with RCA-CVVHD, ROC data supports lactate (cutoff value 2.39 mmol/L) as a strong negative predictive factor of citrate accumulation, with a predictive value of 99.28%27. In our study, lactate has a negative predictive value of 82.1% at a cutoff of 2.65 mmol/L, but its AUROC is only 0.627, and it was not found to be an independent risk factor. These results agree with those of our previous study16. Similarly, the MELD score and PT-INR also showed high negative predictive values but had low AUROCs (AUROCs < 0.750).

In our study, an R-CA score 0–2 predicted the absence of LDCA with a negative predictive value of 90.2%. Approximately 70% of the patients in this study had an R-CA score 0–2. In addition to predicting LDCA, the R-CA score may have the potential to guide initial anticoagulant prescriptions in the future. Patients with an R-CA score 0–2 can safely receive RCA, but the initial dosage should be reduced, and intervention may be needed, in patients with an R-CA score of 3–8. Automated anticoagulant technology may provide a user-friendly and safe system for patients with liver failure treated with RCA-ALSS therapy in the future28.

In this study, we found that R-CA model, R-CA score, and LDCA were positively correlated with disease severity rated by MELD score. As the R-CA model and R-CA score are based on some of the same parameters as the MELD score17, the CLIF-C ACLF score26, and the ACLF criteria19,20, and an elevated MELD score, CLIF-C ACLF score, and Catot/Caion being associated with increased mortality in patients with liver failure17,26,29, the R-CA model, R-CA score, and LDCA could also have the ability to predict patients’ prognosis. The direct relationship between these values and patients’ prognosis requires further investigation.

Our study has several limitations. First, as a retrospective study with a small number of patients from a single geographic area, the patient characteristics may not represent the general population, which may have affected the results of our study. Second, the patients in our study all had chronic HBV infections presenting as acute hepatic decompensation. Data derived from this subset may not be applicable to all patients with liver disease, and requires further investigation. Lastly, we used the Catot/Caion ratio instead of a direct measurement of plasma citrate concentration to reflect citrate accumulation. The upper normal and toxic levels of citrate in the blood are not well established, and as citrate is a physiological metabolite, it may not be toxic, but instead lead to metabolic lag when it accumulates8,15. Further investigation on normal citrate levels, as well as the effect of citrate accumulation in the plasma, is needed.

Conclusions

Our R-CA model, based on gender, PT-INR, serum creatinine and serum chloride, reliably predicts LDCA in patients with HBV-ACLF who are undergoing RCA-ALSS therapy. The simplified R-CA score is also reliable, and easy to use. Patients with an R-CA score 0–2 can safely receive RCA-ALSS therapy, while others should be carefully evaluated and monitored during treatment. Our model requires further validation via prospective cohort studies.

Methods

Study design and patients

Patients treated with ALSS therapy were recorded in a previously established clinical database from the Center of Infectious Diseases, West China Hospital of Sichuan University since January 2014. All ALSS therapies were evaluated on a case-by-case basis by treating physician referring to the Guideline for Diagnosis and Treatment of Liver Failure drawn up by Chinese Medical Association30. Three main indications for ALSS therapy, liver failure or pre-liver failure, severe hyperbilirubinemia with no response to medicine, perioperative period of liver transplantation for end-stage liver disease, are recommended. RCA would be implemented if there were no contraindications (circulatory shock, or hypoxemia that cannot be corrected by oxygen therapy) other than abnormal liver function since January 2018. Patients treated with ALSS therapy between January 2018 and December 2019 were retrospectively included in this study (N = 480; Fig. 1). Patients treated with non-DPMAS plus PE therapy (N = 7) or non-RCA (N = 9) were excluded from the study. Patients with liver cancer (N = 18) and those without HBV infection (N = 108) were also excluded. The remaining 338 patients were randomly divided into two cohorts: a derivation cohort (N = 230), used to develop our R-CA model; and a validation cohort (N = 108), used to test our R-CA model. All patients were followed up 2 h after RCA-ALSS therapy. Approval for this study was obtained from the Biomedical Research Ethics Committee of West China Hospital of Sichuan University. All study components were performed according to the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments. Informed consent was obtained from all subjects or, if subjects were under 18, from a parent and/or legal guardian.

RCA-ALSS therapy

All patients received standard medicinal treatment and DPMAS plus PE therapy. The composition of DPMAS plus PE therapy was the same as that previously described (Fig. 7)15,16. All patients received DPMAS therapy for 2 h, followed immediately by PE therapy with half the total plasma volume (approximately 1,500 mL) for approximately 1 h. As shown in Fig. 7, the parameters of the CRRT machine were set to a blood flow of 130 mL/min, a plasma separation flow of 1,500 mL/h, and a plasma return flow of 1,500 mL/h. Patients received 4% citrate serum sodium anticoagulation with an initial speed of 120 mL/h in the arterial segment during DPMAS plus PE therapy and received 10% calcium gluconate supplement in the venous segment with a speed of 13 mL/h during DPMAS therapy and 74 mL/h during PE therapy.

Figure 7
figure 7

Schematic diagram of the artificial liver support system therapy with regional citrate anticoagulation. Circuit 2 therapy is initiated immediately when circuit 1 therapy is completed.

Full size image

Definition

Under normal conditions, citrate’s half-life is approximately 5 min and citrate is metabolized completely within 30 min of discontinuing a citrate infusion31,32. Here, we defined LDCA as the presence of citrate accumulation 2 h after RCA-ALSS we defined longer duration of citrate accumulation (LDCA) as the presence of citrate accumulation 2 h after RCA-ALSS therapy concluded. Citrate accumulation was defined as the value of the ratio of Catot to Caion greater than or equal to 2.5 (Catot/Caion ≥ 2.5)8,33.

Statistical analysis

Patients were randomly divided into a derivation cohort and a validation cohort using SPSS software (IBM SPSS) with a ratio of 2:1. The t test and the U test were performed for quantitative data of normal distribution and that of non-normal distribution, respectively. The chi-squared test or Fisher’s exact test was performed to calculate differences between qualitative data. The predictors for LDCA in the derivation cohort were analyzed by logistic regression in univariate analysis. For any variables with p ≤ 0.1 in the univariate analysis, the backward stepwise (likelihood ratio) method was performed in a multivariate analysis. The predictors obtained from the derivation cohort were then tested in the validation cohort. The predictive model was also tested in the validation cohort. Predictive factors with an AUROC > 0.750 in the derivation cohort that was equivalent or greater in the validation cohort was used to derive our predictive R-CA model. With the purpose of deriving a simple, specific predictive score for patients treated with RCA-ALSS therapy, we included clinically relevant characteristics and laboratory parameters observed at baseline. An ordinal grading (0–2) was performed for individual parameters by comprehensively considering their cut-off values of AUROCs predicting the probability of LDCA, ACLF diagnostic criteria19,20, and clinically significant values. A score was obtained by combining the individual grade of all the significant parameters. Multiple comparisons of the score with other predictors were performed by AUROC. Statistical significance was set at p < 0.05. The statistical tests were performed using SPSS v.24 (IBM SPSS), except multiple comparisons of AUROCs, which were performed using MedCacl v.19 (MedCalc Software). The figures of AUROCs and linear regression lines were drawn using MedCacl v.19 (MedCalc Software) and GraphPad Prism 6 (GraphPad Software Inc.), respectively.