Abstracts

Accuracy of the GRACE and TIMI scores in predicting the angiographic severity of acute coronary syndrome

Carolina Esteves Barbosa; Mateus Viana; Mariana Brito; Michael Sabino; Guilherme Garcia; Mayara Maraux; Alexandre Costa Souza; Márcia Noya-Rabelo; J. Péricles Esteves; Luis Cláudio Lemos Correia

Escola Bahiana de Medicina e Saúde Pública; Hospital São Rafael; Hospital Português, Salvador, BA - Brazil

^{Mailing Address} Mailing Address: Luis Cláudio Lemos Correia Av. Princesa Leopoldina, 19/402, Graça Postal Code 40150-080, Salvador, BA - Brazil E-mail: lccorreia@cardiol.br; lccorreia@terra.com.br

ABSTRACT

BACKGROUND: The accuracy of the GRACE and TIMI scores in predicting coronary disease extension in patients with non-ST-elevation acute coronary syndromes (ACS) has not been established.

OBJECTIVE: To assess the hypothesis that the GRACE and TIMI risk scores satisfactorily predict coronary disease extension in patients with non-ST-elevation ACS undergoing coronary angiography.

METHODS: Individuals meeting the objective criteria for ACS and undergoing coronary angiography during hospitalization were consecutively assessed. Angiographic coronary disease was described as follows: quantification of coronary disease extension by using Gensini score; presence of any coronary artery obstruction (> 70% or > 50% when affecting left main coronary artery); and presence of severe disease (three-vessel disease or affecting the left main coronary artery).

RESULTS: Of 112 patients assessed, a positive correlation of the Gensini score was observed with the GRACE (p = 0.017) and TIMI (p = 0.02) scores, but that association was weak (r = 0.23 and r = 0.27; respectively). The GRACE score could predict neither obstructive coronary disease (area under the ROC curve = 0.57; 95% CI = 0.46 - 0.69), nor severe coronary disease (ROC = 0.59; 95% CI = 0.48 - 0.70). The TIMI score proved to be a modest predictor of coronary disease (ROC = 0.65; 95% CI = 0.55 - 0.76) and of severe coronary disease (ROC = 0.66; 95% CI = 0.56 - 0.76).

CONCLUSION: (1) There is a positive association between the values of the TIMI or GRACE scores and the extension of coronary artery disease in patients with ACS; (2) however, the degree of that association is not sufficient to make those scores accurate predictors of coronary angiography results.

Keywords: Acute coronary syndrome / complications; propensity score; coronary angiography / utilization; coronary vessels / anatomy & histology.

Introduction

Individuals hospitalized due to non-ST-elevation acute coronary syndromes (ACS) have a wide variation in their disease severity, which ranges according to clinical and laboratory characteristics¹. Thus, risk stratification is essential for adequate clinical decision, discriminating individuals who benefit from more aggressive strategies. The use of multivariate models in the form of scores has been shown to represent the most accurate way to predict risk, being superior to the subjective clinical impression². The TIMI and GRACE risk scores are the most used, and their prognostic value has been established by prospective cohort studies^1,3.

In addition to prognostic assessment, predicting the anatomical extension of coronary artery disease is potentially useful for clinical decision. This is due to the fact that the prediction of obstructive disease (or severely obstructive disease) represents one more criterion in favor of performing coronary angiography, because that finding anticipates the need for myocardial revascularization. Some studies have shown an association between the TIMI score value and the number of arteries affected by obstructive disease. However, none of those studies has performed an analysis of discriminatory accuracy, and, thus, the diagnostic value of that score regarding the presence of coronary artery disease and its severity remains uncertain^4-7. Regarding the GRACE score, no study has assessed its association with coronary artery anatomy.

Aiming at testing the hypothesis that the TIMI and GRACE risk scores are accurate predictors of the presence and extension of obstructive coronary artery disease, this study analyzed patients consecutively admitted to the Acute Coronary Syndromes Registry (Resca), who underwent coronary angiography and had not undergone revascularization surgery. The predictive capacity of the scores was analyzed regarding the following: the quantitative measure of coronary artery disease extension (Gensini score); the presence of any obstructive coronary artery disease; and the presence of severe obstructive coronary artery disease.

Methods

Sample selection

Individuals consecutively admitted to the Coronary Unit of our hospital from August 2007 to January 2009, diagnosed with unstable angina or non-ST-segment elevation myocardial infarction, were considered candidates to Resca. The inclusion criterion in that registry has been defined as typical chest discomfort at rest for the past 48 hours, associated with at least one of the following characteristics: 1) positive myocardial necrosis marker, defined as troponin T > 0.01 ug/L or troponin I > 0.034 µg/L, corresponding to values over the 99^th percentile^8,9; 2) ischemic electrocardiographic changes, consisting of T wave inversion (> 0.1 mV) or transient ST-segment depression (> 0.05 mV); 3) coronary artery disease previously documented, defined by a history of myocardial infarction or previous angiography showing coronary artery obstruction > 50%. For the current analysis, of the patients included in Resca, those undergoing invasive coronary angiography during hospitalization were selected.

GRACE and TIMI scores

For calculating the scores, the following data were used: clinical data on the occasion of patient's presentation to the emergency service; electrocardiographic records of the first six hours of medical care; troponin T measures regarding the first 12 hours of medical care; and the first plasma creatinine value. Elevation of the myocardial necrosis marker as a component of the scores was defined as troponin T > 0.01 ug/L or troponin I > 0.034 µg/L, that is, above the 99^th percentile^8,9. Killip classification¹⁰ has also been applied to patients with unstable angina so that the GRACE score could be calculated.

The criteria previously defined in the respective validation studies of those scores were used. Briefly, the TIMI risk score consists of seven dichotomous variables. The presence of each variable adds one point to the total score, which ranges from zero to seven. Such variables relate to the clinical presentation of ACS (ST-segment depression, elevation in myocardial necrosis marker, more than one angina episode in 24 hours) or the previous characteristics of the patients (age > 65 years, use of aspirin, coronary artery obstruction > 50%, at least three risk factors for atherosclerotic disease)¹.

The GRACE score consists of eight variables, five of which are computed semi-quantitatively, that is, with a different weight for each stratum of age, systolic blood pressure, heart rate, plasma creatinine and Killip class; the other three are computed dichotomously (ST-segment depression, elevation in myocardial necrosis marker, cardiac arrest on admission). The final score can range from zero to 372³.

Assessment of coronary artery disease extension

Coronary artery anatomy was assessed by one single experienced cardiac catheterization physician, who defined the coronary artery disease extension quantitatively and categorically. Quantitative analysis was performed by using the Gensini score^11,12. Briefly, that score assesses 28 coronary artery segments, which are scored according to their anatomical importance (ranging from 0.5 to 5) multiplied by the score regarding the maximum degree of obstruction (0-25% = 2; 26%-50% = 4; 51%-75% = 8; 76%-90% =16; 91%-99% = 32; 100% = 64 points). The points of the 28 segments are summed to yield a final score.

From the categorical viewpoint, coronary artery anatomy was defined as follows: presence of obstructive coronary artery disease (any obstruction > 70% or > 50% if in the left main coronary artery); presence of severe coronary artery disease (obstruction > 70% in the anterior descending, circumflex and right coronary artery, characterizing a three-vessel pattern, or obstruction > 50% in the left main coronary artery).

Data analysis

Considering that the Gensini score showed non-normal distribution (Kolmogorov-Smirnov test), the statistical analyses were mainly non-parametric. Aiming at assessing the association between the GRACE or TIMI risk scores and coronary artery disease extension, some analyses were used. First, the linear association of each risk score with the Gensini score was assessed by use of Spearman correlation, while linear regression was used to demonstrate the strength of the influence of the GRACE and TIMI score values on coronary artery disease extension. Second, the Kruskal-Wallis test was used to compare the Gensini score values between the groups divided according to the tertiles of risk scores. Third, the receiver-operating characteristics (ROC) curve was used to test the predictive accuracy of risk scores regarding the presence of coronary artery disease and of severe coronary artery disease. Significant prediction occurred when the area under the ROC curve was statistically different from 0.5.

The sample size necessary for the correlation analyses and description of the areas under the ROC curve was estimated. For the first analysis, considering a minimum correlation coefficient of 0.25 and alpha of 0.05, a sample size of 96 patients would suffice to provide a statistical power of 80% in the null hypothesis rejection of lack of correlation. In the second analysis, considering an area under the ROC curve of 0.65, 100 patients (30 without disease and 70 with obstructive disease or 70 without severe disease and 30 with severe disease) would be necessary to provide a precision of ± 0.12 in the confidence interval (CI) of the ROC curve, which would suffice to reject the null hypothesis of the 0.50 area.

The scores were described as median and interquartile interval (IQI), and statistical significance was defined as p < 0.05. The SPSS Statistical Software (9.0 version, SPSS Inc., Chicago, Illinois, USA) was used for data analysis.

Results

Characteristics of the sample

During the study, 241 patients were admitted to the Resca, 76 of whom did not undergo coronary angiography during hospitalization and 53 had already undergone surgical myocardial revascularization. Thus, this study assessed 112 patients (mean age of 70 ± 12 years; 51% of the male sex), 63% of whom diagnosed with non-ST-elevation acute myocardial infarction and the others with unstable angina. The median of the GRACE score was 115 (IQI: 91 - 141), while that of the TIMI score was 3 (IQI: 2 - 4), indicating a symmetrical distribution of individuals in the low-, intermediate- and high-risk groups. Killip class > 1 and systolic dysfunction at least moderate (ejection fraction < 45%) were identified in only 11% and 12% of the patients, respectively. Other clinical characteristics are shown in Table 1.

The analysis of the coronary angiography showed that 71% of the patients had at least one coronary artery obstruction > 70%. Severe coronary artery disease (three-vessel disease or left main coronary artery) was identified in 32% of the sample, two-vessel disease in 18% of the sample, single-vessel disease in 21%, and no coronary artery obstruction in 29% of the sample. The Gensini angiographic score showed median of 103 (IQI: 65 - 155) and non-normal distribution (p < 0.001).

Coronary artery anatomy prediction by use of the GRACE score

Positive linear association was observed between the GRACE and Gensini scores, represented by the regression formula Gensini = 48 + 0.65 x GRACE (p = 0.001). Although that relationship is statistically significant, the Spearman correlation coefficient indicated a weak association (r = 0.23; p = 0.017) - Figure 1A.

The Gensini score had a median of 85 (IQI: 60 - 128) in the first tertile of the GRACE score, a median of 89 (67 - 167) in the second tertile, and a median of 132 (78 - 187) in the third tertile. This tendency towards greater coronary artery disease extension according to the GRACE tertiles did not reach statistical significance (p = 0.099) - Figure 1B.

Despite those tendencies, the GRACE score showed no discriminatory capacity between patients with and without obstructive coronary artery disease, having an area under the ROC curve of 0.57 (95% CI: 0.46 - 0.69; p = 0.22) - Figure 1C. The prevalence of obstructive coronary artery disease was similar in the first, second and third tertiles of the GRACE score (66%, 71%, and 77%, respectively; p = 0.58). Similarly, the GRACE score could not discriminate patients with and without severe coronary artery disease (left main coronary artery or three-vessel disease) - area under the ROC curve of 0.59 (95% CI: 0.48 - 0.70; p = 0.13) - Figure 1D. The prevalence of severe coronary artery disease did not significantly differ between the three GRACE tertiles (25%, 33%, and 37%, respectively; p = 0.56).

Prediction of coronary artery anatomy by use of the TIMI score

Positive linear association was observed between the TIMI and Gensini scores, represented by the regression formula Gensini = 79 + 16 x TIMI (p = 0.009). Although that relationship is statistically significant, the Spearman correlation coefficient indicated a weak association (r = 0.28; p = 0.003) - Figure 2A.

The Gensini score had a median of 78 (IQI: 62 - 117) in the first tertile of the TIMI score, a median of 107 (62 - 188) in the second tertile, and a median of 120 (95 - 166) in the third tertile (p = 0.024) - Figure 2B.

The TIMI score showed a modest discriminatory capacity between patients with and without obstructive coronary artery disease, having an area under the ROC curve of 0.65 (95% CI: 0.55 - 0.76; p = 0.01) - Figure 2C. The prevalence of obstructive coronary artery disease had a tendency towards an increase according to the ascending TIMI tertiles (61%, 69%, 86%; p = 0.056). Similarly, the TIMI score discriminated modestly patients with or without severe coronary artery disease (left main coronary artery or three-vessel disease) - area under the ROC curve of 0.66; 95% CI: 0.56 - 0.76; p = 0.008 - Figure 2D. That modest discriminatory capacity reflected in a lower prevalence of severe disease in the first GRACE tertile as compared with the two higher tertiles (15%, 44%, 40%, respectively; p = 0.10).

Discussion

The present study does not confirm the initial hypothesis that the TIMI and GRACE risk scores predict accurately the presence and anatomical extension of coronary artery disease in patients with ACS.

In this study, coronary anatomy was described as a quantitative variable (Gensini score) and as a categorical variable (presence of obstructive or severe disease). In the first case, a statistically significant correlation of the coronary anatomy with both scores was observed. That tendency is confirmed by the dose-response gradient between the tertiles of the scores and the value of the Gensini score, statistically significant in the case of the TIMI score and borderline in the case of the GRACE score. Such analyses suggest that the prognostic information contained in the scores is partially mediated by the extension of coronary anatomy.

However, when analyzing the strength of the association of those scores with coronary anatomy, the results differ. First, the weak correlation (r < 0.30) with the Gensini score showed that the calibration of the prognostic scores to coronary anatomy is small. That is, using the regression formula for coronary artery disease extension, the difference between the predicted and the observed values of the Gensini score is large. Second, there is a reasonable overlap of the IQI of the Gensini score between the tertiles of the GRACE or TIMI scores. Third, the area under the ROC curve of the GRACE score is not statistically significant and that of the TIMI score, although statistically significant, shows modest accuracy in discriminating patients with obstructive disease or with severe disease (area < 0.70). Such data indicate that the prognostic scores do not play a good role as predictive models of coronary anatomy.

The prognosis of coronary artery disease is known to be directly influenced by the extension of arterial impairment¹². How to explain that the prognostic models are not good predictive models of the anatomy? The response to that question originates from the fact that the disease anatomic extension represents only one of the several determinants of severity in a complex multivariate model, typical of most biological systems. That is, prognosis does not depend only on anatomy. Thus, those scores can predict risk, without necessarily predicting anatomy at the same proportion. For example, even having single-vessel disease, a patient can have an infarction with severe left ventricular dysfunction in the presence of proximal impairment of the descending anterior coronary artery, causing severe extensive ischemia. The severity of that patient would reflect in the hemodynamic and electrocardiographic variables, and in the necrosis markers that constitute those scores.

The GRACE score has shown a greater prognostic value as compared with that of the TIMI score in different studies^2,5,13. Paradoxically, the TIMI score has shown better prediction of the anatomy. This finding can be explained by the fact that the TIMI score considers some variables specifically related to the existence of coronary artery disease¹. For example, one of the variables of TIMI score is the patient's previous knowledge about his/her obstructive coronary artery disease; in addition, previous aspirin use and number of risk factors, variables exclusive to the TIMI score, indicate more extensive coronary artery disease.

The practical implications of the results found should be discussed. Both scores are commonly used for the clinical decision of patients with ACS, aiming at defining the aggressiveness of the antithrombotic treatment and the either invasive or selective strategy of stratification^14,15. That use is based on the established prognostic value of those scores and on the relationship between baseline risk and the magnitude of the benefit of certain strategies. It is worth noting that our results do not contradict the value of those scores for clinical decision. However, we should recognize that they would be more useful if they could predict coronary anatomy. This is due to the fact that the invasive strategy, applied to patients at medium to high risk according to those scores, is completed with a procedure of revascularization of the culprit artery. If the scores could better identify the presence of obstructive disease, they would identify patients to whom the invasive strategy would be more useful. Another example of usefulness would be the prediction of patients with extensive disease, preventing the administration of certain antiplatelet agents that would increase their risk of bleeding in an occasional early surgery.

Regarding originality, this is the first study describing the association between the GRACE score and coronary anatomy. Regarding the TIMI score, four studies have assessed that question^4-7. All have reported a positive association between score value and coronary artery disease extension. However, we know that the association is not sufficient to ensure accuracy, which should be assessed by use of tests of calibration and discriminatory capacity of ill individuals versus healthy individuals (or severe disease versus non-severe disease). Because ours was the only study to perform that type of analysis, our conclusion contradict the TIMI score value in predicting anatomy, differing from the previous studies which were limited to demonstrating the presence of the association between the TIMI score and the number of arteries affected. Our data confirm the association with disease extension, but demonstrate that it is insufficient to provide satisfactory accuracy. In addition, of the studies cited, ours was the first to measure the coronary artery disease extension quantitatively by using the Gensini score.

This study had limitations, such as the selection bias regarding its target population, individuals with non-ST-elevation ACS. Ideally, we should assess the entire sample of patients meeting diagnostic criteria for that clinical condition. However, only 60% of the individuals with no previous surgery underwent coronary angiography, limiting the study sample. Although that is a characteristic inherent to most studies about coronary anatomy, it may cause a bias in the assessment of accuracy of risk scores regarding anatomy. In practice, patients with low scores tend to undergo coronary angiography less frequently. Such bias reduces the specificity of the risk score in recognizing patients without obstructive coronary artery disease. That is, those scores might identify a certain number of patients as having neither obstructive disease nor severe disease, but they were not computed in the analysis, because the reference standard (coronary angiography) was not performed. However, sensitivity can be overestimated, because the group of patients undergoing coronary angiography tends to have higher score values more frequently. That phenomenon is called assessment bias¹⁶. Thus, we avoided describing specifically sensitivity and specificity, choosing to analyze the area under the ROC curve, which would represent a joint analysis of those two diagnostic properties¹⁷.

Our second limitation is the relatively small sample size, which was partially overcome through the data analysis approach. First, the quantitative description of coronary artery disease (Gensini score) provided sufficient statistical power to the correlation analysis. Second, when describing the area under the ROC curve, the sample size was sufficient to ensure satisfactory precision. However, it is worth noting that future studies with a higher number of patients will provide more precision to our estimates.

In conclusion, the present study suggests that the TIMI and GRACE scores are not accurate tools to predict the coronary artery disease extension in patients with non-ST-elevation ACS. The usefulness of those scores for clinical reasoning should be limited to estimating the probability of recurring cardiovascular events.

Potential Conflict of Interest

No potential conflict of interest relevant to this article was reported.

Sources of Funding

This study was partially funded by Fundação de Amparo à Pesquisa da Bahia (FAPESB).

Study Association

This article is part of the thesis of master submitted by Carolina Esteves Barbosa , from Escola Bahiana de Medicina.

References

Manuscript received November 16, 2011

Manuscript revised November 16

Accepted April 5, 2012

Publication Dates

History