Validity of an automated algorithm using diagnosis and procedure codes to identify decompensated cirrhosis using electronic health records

Introduction

Complications of liver cirrhosis include an increased risk of hepatocellular carcinoma (HCC), liver transplant, and death from liver failure.^1–3 In “compensated” cirrhosis, the damaged liver still functions adequately; cirrhosis that has progressed sufficiently to interfere with essential bodily functions is classified as “decompensated”. The development of clinical decompensation represents a milestone event for the patient with cirrhosis, and is associated with increased morbidity, mortality, and utilization of health care resources—yet there remains uncertainty regarding precisely how to define this important phenomenon.

Although liver biopsy is considered the gold standard for confirming the presence of cirrhosis, it cannot be used to determine hepatic functional status.^4,5 Measurement of hepatic venous pressure gradient has been shown to be an excellent predictor of stage of fibrosis and risk for decompensated cirrhosis (DC), but the procedure carries risks and is not commonly performed in clinical practice. While the Child-Pugh classification stages severity of cirrhosis and the Model for End-Stage Liver Disease (MELD) scores predict survival in a patient with cirrhosis, there remains considerable ambiguity regarding whether a cirrhotic patient has experienced decompensation.⁶ Signs and symptoms (including past or present ascites, encephalopathy, and even gastrointestinal bleeding) represent continuums that may be variously interpreted. In clinical trials, for example, decompensated cirrhotics are categorized as distinct cohorts. An efficient system to identify decompensated patients from administrative data may inform early-intervention clinical trials or epidemiological research to reduce poor outcomes.

Epidemiological, outcome, and economic studies of viral hepatitis often rely on International Classification of Diseases, 9th (or 10th) Revision (ICD-9/10) and Current Procedural Terminology (CPT) codes obtained from administrative, claims, or electronic health record (EHR) databases to identify patients with cirrhosis and decompensation.^3,7,8 However, diagnostic and procedure code data are subject to limitations, including coding inaccuracy and incompleteness. Several studies have been performed in United States Veterans Administration populations^1,9,10 to evaluate the validity of diagnosis codes for identification of cirrhosis/DC (alone or in combination with laboratory markers); however, these studies were limited by use of a restricted patient population, inconsistent results, or a focus on risk factors associated with DC rather than disease classification. A more recent study in a civilian population validated a model for identifying cirrhosis (a classification study) based on diagnosis codes and laboratory abnormalities.¹¹ However, the outcome in this study was “cirrhosis”, not specifically decompensation; likewise, laboratory values are not typically available in administrative and claims databases (for example, EHR data). Therefore, we sought to develop and optimize a model based exclusively on electronic data-derived ICD-9 and CPT codes to accurately identify patients with DC in a civilian cohort of viral hepatitis patients.

Materials and methods

Study design

Our sample was drawn from a subset of hepatitis patients from the Chronic Hepatitis Cohort Study (CHeCS), an observational study of patients from four large US health care systems, including Henry Ford Health System (HFHS, Detroit, MI). The study follows all guidelines of the US Department of Health and Human Services regarding the protection of human subjects; our protocol was approved and renewed annually by the Institutional Review Boards of HFHS; Geisinger Health System; Kaiser Permanente Northwest; and Kaiser Permanente Hawai’i. Due to the de-identified nature of this observational study, requirements for written informed consent were waived. Patient data were anonymized and de-identified prior to analysis. HFHS CHeCS patients were considered for study inclusion if they had biopsy-confirmed or excluded cirrhosis and clinical encounters data collected through the earlier date of liver transplant (if applicable) or December 31, 2010. In patients with multiple biopsies, the most severe biopsy stage was used. A random sample of 296 patients with a 2:1 ratio of cirrhosis and noncirrhosis was selected using SAS.¹² Harrell’s 1-to-10 rule of thumb¹³ was used to estimate sample size; assuming that 50% of cirrhosis cases were decompensated, up to 10 binary/continuous predictors for multivariable modeling could be analyzed without overfitting the data.

Chart reviews were performed to classify the patients into one of three categories: 1) noncirrhotic (NC); 2) compensated cirrhotic (CC); and 3) decompensated cirrhotic (DC). To ensure consistency, a senior hepatologist (SG) provided standard diagnosis guidelines and two gastroenterology fellows (“raters”) performed the chart reviews. Confirmation of at least one of the following conditions during chart review resulted in a diagnosis of DC: hepatic encephalopathy; gastrointestinal bleeding due to portal hypertension; jaundice with a total bilirubin of >2.5 mg/dL; ascites/hydrothorax due to portal hypertension; or HCC. The two raters independently reviewed all patient charts. Weighted kappa (κ) was used to measure the agreement between the raters, accounting for the ordinal property of cirrhosis levels, where κ >0.80 indicates “excellent” agreement. Any discrepancies in classification between the raters were reviewed by the senior hepatologist for adjudication. Final clinical classification of DC versus compensated cirrhosis (CC) or noncirrhosis was used as the outcome of interest.

We hypothesized that a set of diagnosis and procedure codes could identify DC. To begin, we modified a set of 41 ICD-9 and CPT codes previously used by Spradling et al⁷ to identify cirrhotic status. These codes were grouped into ten clusters related to clinical presentation of hepatic decompensation (Table 1). The data collected from each cluster were either quantitative (counts) or categorical (presence/absence). In the previous study,⁷ a three-level cirrhosis classification variable (NC, CC, and DC) was derived from ten disease conditions (clusters). The variable was defined as “NC” if no ICD-9/CPT codes occurred in any of the C1–10 cluster categories, as “CC” if any codes from the C10 cluster were present and codes from the C1–C9 clusters were absent, and as “DC” if any codes from the C1–C9 clusters were present. We first validated this three-level cirrhosis classification variable, then developed and optimized models to predict DC based on the individual C1–C10 cluster variables for greater accuracy.

Table 1 ICD-9/10 and CPT codes comprising the 10 cluster variables (C1–C10) Abbreviations: CPT, Current Procedural Terminology; ICD, International Classification of Diseases.

Statistical analysis

Descriptive statistics were used to summarize characteristics of the study population. Mean and standard deviation were used for continuous variables; frequency and percentage were used for categorical variables. Two novel classifiers were considered for the classification of DC; the first was logistic regression using the Least Absolute Shrinkage and Selection Operator (LASSO)¹⁴ approach for variable selection that yields a variable combination with optimal model predictive ability. The second model used Classification and Regression Trees (CART),^15,16 a nonparametric approach that generates a binary recursive multilevel tree that partitions the variables most predictive of the outcome of interest (in this case, DC).¹⁴ The CART method is ideally suited for clinical decision making because it can reveal important relationships between variables that may remain hidden when using logistic regression.^15,16 Finally, a hybrid method between generalized LASSO and CART was used to simplify the model for clinical interpretation without loss of generalizability and accuracy.

Model learning and testing were performed based on the sample of 296 patients. Tenfold cross-validation was performed and model predictive accuracy was based on validation results. Predictive accuracy of the model was assessed using area under receiver operating characteristic (AUROC) curves, as well as positive predictive value (PPV), sensitivity, and specificity. Models are considered to have “reasonable” to “excellent” predictive ability when the AUROC is 70%–80% and 80%–90%, respectively. The nonparametric Mann–Whitney approach¹⁷ was used to compare the difference in AUROC between models. All analyses were performed using SPM version 8.0 (Salford Predictive Modeler)¹⁴ and SAS version 9.4.¹²

Given the ongoing implementation of ICD version 10, we conducted an ad hoc analysis based on both ICD-9 and ICD-10 liver-related diagnosis and procedure code data collected from 01/01/2015 to 10/01/2015, and compared C1–10 cluster variables as defined by both ICD-9 and ICD-9/10 codes. The detailed cluster variables with related ICD-10 codes are illustrated in Table 1.

Results

Demographic characteristics are presented in Table 2. Two physician raters completed independent chart reviews to classify 296 patients (23 hepatitis B virus [HBV] infected, 268 hepatitis C virus [HCV] infected, and 5 co-infected). Agreement between our two raters was “excellent” (κ=0.88; 95% Confidence Interval (CI): 0.84–0.92), indicating consistent and reliable classification assessment.¹⁸ Eighteen inconsistent classifications were further reviewed by the senior hepatologist (SG). Based on this chart review, 102 (34%) patients did not have cirrhosis, 67 (23%) had CC, and 127 (42%) had DC. The previously defined three-level cirrhosis classification variable resulted in an AUROC of 80%, PPV of 70%, sensitivity of 90%, and specificity of 70% (data not shown).

Table 2 Study population characteristics Notes: aCurrency is US$.

LASSO model

The inclusion of the full set (C1–C10) of ICD-9 variable clusters in the initial multivariate logistic LASSO regression model resulted in five retained clusters (C1, C2, C6, C8, and C10). Based on this model, the likelihood of DC can be predicted by a linear combination of the five cluster indicators: −1.52+0.98×C1+1.17×C2+0.98×C6+1.25×C8+0.48×C10. Tenfold cross-validation estimated an AUROC of 89%, PPV of 83%, sensitivity of 83%, and specificity of 88%, using an optimized cutoff of 2.28 (Table 3A). The same predictive ability was observed when the cluster variables were quantitative (counts) or categorical (presence/absence).

Table 3 Classification models for identification of DC Abbreviations: C1, liver transplant; C2, liver cancer; C6, esophageal varices complications (bleeding) and procedures; C8, ascites/paracentesis procedures; C10, cirrhosis; AUROC, area under receiver operating characteristic; CART, Classification and Regression Trees; DC, decompensated cirrhosis; LASSO, Least Absolute Shrinkage and Selection Operator; PPV, positive predictive value.

For clinical applications, we then developed a single, simplified five-cluster indicator combination model, Z₁=C1+C2+C6+C8+C10, with Z₁ ranging from 0 to 5, where higher scores indicate a greater likelihood of DC. The overall prediction accuracy (AUROC) remained similar, at 90% with tenfold validation (Table 3B). There was no difference between the AUROCs of the original and simplified models (p=0.09).

CART model

CART analysis generated a recursive tree with three cluster indicator variables (C2, C6, C8) and four terminal nodes (TN) (Figure 1). The AUROC was 86.8% using the learning data and 85.8% using the testing data (ten-fold validation). As illustrated in Figure 1, presence of HCC (C2) first identified 94 patients as decompensated (TN4, with 20 misclassified, 21%); patients without HCC would be identified as decompensated if there was indication of ascites (C8; n=35 in TN3 with 5 misclassified, 14%); patients without either HCC or ascites would be identified as decompensated if they had bleeding esophageal varices (C6; n=6 in TN2, with no misclassifications); PPV, sensitivity, and specificity were 81%, 87%, and 85%, respectively, for both the learning data and testing data (Table 3C). Notably, the three cluster variables included in the final CART model were also identified in the LASSO regression model.

Figure 1 The three cluster-variable CART model. Abbreviations: C1, liver transplant; C2, liver cancer; C6, esophageal varices complications (bleeding) and procedures; C8, ascites/paracentesis procedures; C10, cirrhosis; CART, Classification and Regression Trees; DC, decompensated cirrhosis; TN, terminal node.

By using the CART approach with the simplified five-cluster combination variable (Z₁) added to the ten cluster indicators (a hybrid method between LASSO and CART), only Z₁ remained in the final CART model (Figure 2). A cutoff value of Z₁≥2 (indicating presence of codes from two or more of the five cluster variables) had a validated AUROC of 86%, PPV of 82%, sensitivity of 85%, and specificity of 86% (Table 3D).

Figure 2 The hybrid CART model. Abbreviations: C1, liver transplant; C2, liver cancer; C6, esophageal varices, complications (bleeding), and procedures; C8, ascites/paracentesis procedures; CART, Classification and Regression Trees; DC, decompensated cirrhosis; TN, terminal node.

In addition, a total of 2177 patients (1982 HCV, 116 HBV, and 18 HBV/HCV coinfected) were considered who had at least one of the C1–C10-related diagnosis codes from 01/01/2015 to 10/01/2016. Of them, 125 patients had C1–C10-related ICD-10 diagnosis and procedures. There was complete concordance between ICD-9 and ICD-9/10 classifications for all patients except one, who presented with C2 (HCC)-related ICD-10 codes.

Discussion

In clinical practice, the terms “decompensatd cirrhosis” and “end-stage liver disease” lack precise definitions. Moreover, patients with cirrhosis may experience “episodes of decompensation”, such as transient ascites formation, and then revert back to a fully compensated state once ascites resolves. Similar events of variceal bleeding or hepatic encephalopathy, for example, may represent transient phenomena. With the advent of newer direct-acting all-oral (DAA) treatments for HCV, it is especially important to have consensus regarding this clinically important condition, because treatment options may vary if a cirrhotic patient has ever experienced “decompensation”. For example, the US Food and Drug Administration recently issued a warning regarding the use of protease inhibitor antiviral therapy in chronic HCV patients who have ever experienced hepatic decompensation.

Our method—based on EHR-based diagnosis and procedure codes—may provide both clinicians and researchers an efficient and accurate method for the identification of decompensation among patients with viral hepatitis. Due to the low accuracy of individual diagnosis codes in detecting DC,⁷ we grouped these codes into clusters of diagnoses related to specific manifestations of decompensation. We first validated a previously defined three-level classification variable based on ten clusters of 41 ICD-9 and CPT codes. We then developed an optimal algorithm based on the individual C1–C10 cluster variables for greater predictive ability.

To ensure the reliability of our classification of DC, medical records were reviewed by two trained raters and discrepancies were adjudicated by an expert hepatologist; agreement between the reviewers was “excellent” (κ=0.88; 95% CI: 0.84–0.92), indicating high reliability. Our results showed that the predictive ability of the previously defined three-level cirrhosis classification variable (from 41 diagnosis and procedure codes) was “excellent” (AUROC=80%), although specificity was only “moderate” (70%), indicating a high false-positive rate.

The parametric LASSO regression model (−1.52+0.98×C1+1.17×C2+0.98×C6+1.25×C8+0.48×C10; Table 3A), based on five cluster variables derived from 27 individual diagnosis and procedure codes, demonstrated the likelihood (probability) of decompensation on a continuous scale. The model’s predictive ability was in the upper range of “excellent” (AUROC=89%; PPV=83%). The simplified LASSO regression model (Table 3B) using a coefficient of 1 for all five cluster variables (Z₁=C1+C2+C6+C8+C10) retained similar identification accuracy (AUROC=86%; PPV=82%). Notably, the nonparametric CART model ended with the same five cluster variables (Table 3C) obtained from the LASSO model, which indicates robustness of the DC identification process.

The classification model using a hybrid LASSO/CART approach with a cutoff of Z₁≥2 (Table 3D) retained similar identification accuracy (AUROC=86%; PPV=82%). Cirrhotic patients would be classified as “decompensated” if they ever had one of the following four conditions—liver transplant, HCC, complications or procedures related to esophageal varices, or ascites. This model may be more useful in a clinical setting or for researchers.

The abbreviated three-cluster variable model (derived from 17 individual diagnosis and procedure codes) identified DC (Figure 1) with only slightly reduced predictive accuracy (AUROC=86%; PPV=81%). Patients would be classified as “decompensated” at Level 1 in the presence of HCC-related codes (C2); at Level 2 in the absence of HCC-related codes, but in the presence of ascites-related codes (C8); and at Level 3 in the absence of HCC- and ascites-related codes, but in the presence of codes related to bleeding esophageal varices (C6). The agreement between the hybrid five-cluster variable model with Z₁≥2 as the cutoff and the CART three-cluster models was “outstanding” (κ=0.93; 95% CI: 0.88–0.97). This simplified model would be easy to use in clinical practice: cirrhotic patients may reliably be considered “ever decompensated” if they have one of the following four conditions in their medical records—liver transplant, HCC, complications or procedures related to esophageal varices, or ascites.

We note that our sample included only patients with chronic viral hepatitis. However, diagnosis and procedure codes related to viral hepatitis were not included in the model and, thus, were not relevant to its accuracy. Likewise, the conditions that define cirrhotic decompensation are the same regardless of etiology. This algorithm also requires further validation in an independent cohort with similar patient characteristics, as well as among patients with other causes of liver disease.

Our ad hoc analysis showed that ICD-10 codes enrich ICD-9 codes. In CART models using either the cutoff value of 1.77 or ≥2 C1–C10 clusters, we conclude that our ICD-9–based CART model could be adapted and replaced with ICD-9/10 codes.

In conclusion, our study showed that the presence of diagnosis and procedure codes in patient records could be used to efficiently identify patients with DC. We expect the results of this study to be useful for epidemiological and economic research of viral hepatitis-related liver disease, as well as serve as a quality improvement tool to ensure the best available care for such patients. Our algorithm may also be useful in the identification of risk factors for decompensation in observational studies with large cohorts of viral hepatitis patients. There are now effective therapies that may be life-saving for cirrhotic patients before or even after their condition worsens⁶ —this tool may help clinicians recognize that cirrhosis has evolved to the point of decompensation. Our algorithm could be applied to electronic medical record systems to trigger automated alerts for specialty referrals; such alerts may contribute to timely clinical management of cirrhosis and its complications.

Acknowledgments

The CHeCS Investigators include the following investigators and sites: Scott D Holmberg, Eyasu H Teshale, Philip R Spradling, Anne C Moorman, Jim Xing, and Yuna Zhong, Division of Viral Hepatitis, National Centers for HIV, Viral Hepatitis, STD, and TB Prevention (NCHHSTP), Centers for Disease Control and Prevention (CDC), Atlanta, GA; Stuart C Gordon, David R Nerenz, Mei Lu, Lois Lamerato, Jia Li, Loralee B Rupp, Nonna Akkerman, Nancy Oja-Tebbe, Sheri Trudeau, Yueren Zhou, and Talan Zhang, Henry Ford Health System, Detroit, MI; Joseph A Boscarino, Zahra S Daar, and Robert E Smith, Center for Health Research, Geisinger Health System, Danville, PA; Yihe G Daida, Connie Mah Trinacty, Jonathan W Lai, Carmen P Wong, The Center for Health Research, Kaiser Permanente-Hawaii, Honolulu, HI; Mark A Schmidt and Judy L Donald, The Center for Health Research, Kaiser Permanente-Northwest, Portland, OR, USA.

Financial Support: Henry Ford Health System receives funding for CHeCS from the Centers for Disease Control and Prevention and from Gilead Sciences. CHeCS was previously funded through May 2016 by the CDC Foundation, which received grants from AbbVie; Genentech, A Member of the Roche Group; Gilead Sciences; Janssen Pharmaceuticals, Inc. and Vertex Pharmaceuticals; past partial funders include Bristol-Myers Squibb. Granting corporations do not have access to CHeCS data and do not contribute to data analysis or writing of manuscripts.

Disclosure

Stuart C Gordon receives grant/research support from AbbVie Pharmaceuticals, Bristol-Myers Squibb, Conatus, CymaBay, Exalenz BioScience, Gilead Pharmaceuticals, Intercept Pharmaceuticals, and Merck. He is also a consultant/advisor for AbbVie Pharmaceuticals, Bristol-Myers Squibb, CVS Caremark, Gilead Pharmaceuticals, Intercept, and Merck, serves as a speaker/teacher in programs sponsored by Intercept Pharmaceuticals. The authors report no other conflicts of interest in this work.

References

1.		Goldberg D, Lewis J, Halpern S, Weiner M, Lo Re V 3rd. Validation of three coding algorithms to identify patients with end-stage liver disease in an administrative database. Pharmacoepidemiol Drug Saf. 2012;21(7):765–769.
2.		Gordon SC, Pockros PJ, Terrault NA, et al. Impact of disease severity on healthcare costs in patients with chronic hepatitis C (CHC) virus infection. Hepatology. 2012;56(5):1651–1660.
3.		Holmberg SD, Lu M, Rupp LB, et al. Noninvasive serum fibrosis markers for screening and staging chronic hepatitis C virus patients in a large US cohort. Clin Infect Dis. 2013;57(2):240–246.
4.		Regev A, Berho M, Jeffers LJ, et al. Sampling error and intraobserver variation in liver biopsy in patients with chronic HCV infection. Am J Gastroenterol. 2002;97(10):2614–2618.
5.		Runyon BA; Practice Guidelines Committee, American Association for the Study of Liver Diseases (AASLD). Management of adult patients with ascites due to cirrhosis. Hepatology. 2004;39(3):841–856.
6.		Garcia-Tsao G, Friedman S, Iredale J, Pinzani M. Now there are many (stages) where before there was one: in search of a pathophysiological classification of cirrhosis. Hepatology. 2010;51(4):1445–1449.
7.		Spradling PR, Rupp L, Moorman AC, et al. Hepatitis B and C virus infection among 1.2 million persons with access to care: factors associated with testing and infection prevalence. Clin Infect Dis. 2012;55(8):1047–1055.
8.		Kramer JR, Giordano TP, Souchek J, Richardson P, Hwang LY, El-Serag HB. The effect of HIV coinfection on the risk of cirrhosis and hepatocellular carcinoma in U.S. veterans with hepatitis C. Am J Gastroenterol. 2005;100(1):56–63.
9.		Kramer JR, Davila JA, Miller ED, Richardson P, Giordano TP, El-Serag HB. The validity of viral hepatitis and chronic liver disease diagnoses in Veterans Affairs administrative databases. Aliment Pharmacol Ther. 2008;27(3):274–282.
10.		Lo Re V 3rd, Lim JK, Goetz MB, et al. Validity of diagnostic codes and liver-related laboratory abnormalities to identify hepatic decompensation events in the Veterans Aging Cohort Study. Pharmacoepidemiol Drug Saf. 2011;20(7):689–699.
11.		Nehra MS, Ma Y, Clark C, Amarasingham R, Rockey DC, Singal AG. Use of administrative claims data for identifying patients with cirrhosis. J Clin Gastroenterol. 2013;47(5):e50–e54.
12.		SI Inc. SAS/ATAT Software. Version 9.4. 8th ed. Cary, NC: SAS Institution Inc.; 2013.
13.		Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361–387.
14.		CART 6.0 User’s Guide Salford Systems [computer program]. 2010.
15.		Breiman L, Friedman J, Stone Charles J. Classification and Regression Trees. 1st ed. New York: Chapman and Hall; 1984.
16.		Moore CL, Lu M, Cheema F, et al. Prediction of failure in vancomycin-treated methicillin-resistant Staphylococcus aureus bloodstream infection: a clinically useful risk stratification tool. Antimicrob Agents Chemother. 2011;55(10):4581–4588.
17.		DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–845.
18.		Fleiss JL. The Design and Analysis of Clinical Experiments. New York: John Wiley & Sons; 1986.