Introduction

Primary open-angle glaucoma is a multifactorial optic neuropathy in which there is an acquired loss of retinal ganglion cells and their axons in the retina.1, 2, 3 Damage to the retinal nerve fiber layer (RNFL) is usually followed by changes in the shape of the optic nerve head (ONH) and specific visual field defects. The detection of these changes at the optic disc is key to the diagnosis and follow-up of glaucoma. ONH evaluation in glaucoma management is traditionally based on physician drawings and photographic documentation. ONH evaluation by slit-lamp examination with a high-power bi-convex lens is subjective, however, and stereophotographs, requiring experienced evaluators, are not always taken in clinical practice. Thus, in recent years, several devices designed to quantitatively assess ONH morphology have been developed to avoid subjectivity of evaluators. One of these devices is the optical coherence tomograph, which is a computer-assisted precision optical instrument that delineates cross-sectional anatomy of the retina and the ONH.4, 5

The aim of this study was to optimize the sensitivity–specificity balance of ONH parameters of optical coherence tomography (OCT) by calculating a binary logistic regression analysis. Binomial (or binary) logistic regression analysis is useful for predicting the presence or absence of a characteristic or outcome based on values of a set of predictor variables. This method can be used to find a linear combination of variables whose value is as similar as possible within groups and as different as possible between groups. The linear combination is called a linear discriminant function (LDF). In our study, we used this procedure to determine which parameters are more useful for differentiating between healthy individuals and glaucoma patients with glaucomatous visual field defects.

The design of the study followed the 25 items of the guidelines suggested by the Standards for Reporting of Diagnostic Accuracy initiative6 to increase the quality of reporting diagnostic accuracy research.

To the best of our knowledge, this is the first study to assess the diagnostic ability of an LDF designed for the Stratus OCT based exclusively on ONH parameters. The strength of this study lies in the validation of LDF using an independent sample.7

Materials and methods

The design of the study followed the Declaration of Helsinki Principles. The study protocol was approved by the Clinical Research Ethics Committee of Aragon (Zaragoza, Spain), and informed written consent was obtained from all participants.

Subjects and measurement protocol

Required inclusion criteria were as follows: best-corrected visual acuity of 20/40 or better, refractive error within ±5.00 diopters equivalent sphere and ±2.00 diopters astigmatism, transparent ocular media (nuclear colour/opalescence, cortical or posterior subcapsular lens opacity <1) according to the Lens Opacities Classification System III system,8 and open anterior chamber angle. Exclusion criteria included previous intraocular surgery, diabetes, or other diseases affecting the visual field, and current use of medications that could affect visual field sensitivity.

From April 2008 to September 2008, two independent samples of 280 consecutive healthy individuals and glaucoma patients were prospectively recruited from two outpatient clinics under the area of influence of our hospital. Patients referred for refraction that underwent routine examination without abnormal ocular findings were recruited for normal eye controls. One sample was randomly selected for developing the LDF (teaching set) and the other for checking the performance of the LDF (validating set).

Seven subjects did not complete all of the required tests, and 9 subjects were unable to perform a reliable standard automated perimetry (SAP) after three attempts; these 16 subjects were therefore excluded from further analysis. A total of 264 eyes of Caucasian origin were included in the statistical analysis. One eye from each subject was randomly chosen for the study, unless only one eye met the inclusion criteria.

All participants underwent a full ophthalmologic examination: clinical history, visual acuity, biomicroscopy of the anterior segment using a slit lamp, gonioscopy, Goldmann applanation tonometry, central corneal ultrasonic pachymetry (model DGH 500; DGH Technology, Exton, PA, USA), and ophthalmoscopy of the posterior segment.

At least two reliable SAP tests per eye were performed using a Humphrey Field analyzer, model 750i (Zeiss Humphrey Systems, Dublin, CA, USA), with the SITA Standard 24–2 strategy. If fixation losses were greater than 15% and false-positive or false-negative rates were greater than 20%, the test was repeated. The second reliable perimetry test obtained was used in this study to minimize the learning effect.9 Abnormal SAP results were considered to be a reproducible glaucomatous visual field loss in the absence of any other abnormalities to explain the defect. Visual field loss was defined as a pattern standard deviation significantly elevated beyond the 5% level and/or a Glaucoma Hemifield Test outside normal limits. The subjects completed the perimetry tests before any clinical examination or structural test. Each perimetry test was performed on different days to avoid a fatigue effect.

The Zeiss Stratus OCT 3000 (Carl Zeiss Meditec, Dublin, CA) was used to obtain ONH measurements (software version 4.0.7). The Fast Optic Disc scanning protocol, which comprises six radial scans centred on the ONH, was used to acquire the OCT images. The Optic Nerve Head (Single Eye) analysis protocol was used to analyze the scans.4 The automatically determined disc margin was used in this study to avoid subjectivity in the measurement procedure. Good quality scans were defined as focused images from the ocular fundus that were centred on the optic disc and signal strength of 7 or higher. In all cases, a satisfactory quality image was obtained. All the ophthalmic examinations were performed within 2 months of the subject's date of enrolment into the study.

Classification into groups

Healthy eyes had an intraocular pressure (IOP) of less than 21 mm Hg, no history of increased IOP, and a normal SAP. Glaucomatous eyes had an IOP higher than 21 mm Hg (on at least three readings on different days) and abnormal SAP results, regardless of the appearance of the optic disc. The eyes were classified by two glaucoma specialists masked to patient identity and clinical history.

Statistical analysis

All statistical analyses were calculated using SPSS (version 15.0; SPSS Inc., Chicago, IL, USA) and MedCalc (version 9.6.4.0; MedCalc Software, Mariakerke, Belgium) statistical software. The teaching set was used for binary logistic regression analysis, a regression analysis that is used when the dependent variable is dichotomous (healthy or diseased) and the independent variables are of any type. The dependent variable was glaucoma (yes or no) and the predictive variables were the following ONH parameters measured with OCT: vertical rim area, horizontal rim width, disc area, cup area, rim area, cup/disc area ratio, cup/disc horizontal ratio, and cup/disc vertical ratio.

For a logistic regression, the predicted dependent variable is a function of the probability that a particular subject will be in one of the categories (for example, the probability that one subject has glaucoma, given his set of scores on the predictor variables). The relative importance of each independent variable was assessed by stepwise binary logistic regression analysis using the forward Wald method. The Wald χ2-statistic tests the unique contribution of each predictor in the context of the other predictors (holding constant the other predictors), eliminating any overlap between predictors. Hence, the parameters with higher sensitivity/specificity values do not necessarily have to be the selected variables in the logistic regression method. The stepwise probability test determined the criteria by which variables were entered into and removed from the model. The LDF was a score calculated by taking the weighted sum of the predictor variables. The significant OCT parameters were combined to generate a new variable (the LDF) in such a way that the measurable differences between healthy and glaucoma eyes were maximized. The validating set was used to test and compare the diagnostic accuracy of our LDF with other ONH parameters of the OCT. The receiver operating characteristic (ROC) curves were plotted for all of the parameters and compared with the proposed LDF. Differences between the ROC curves were tested to compare the area under the ROC curves (AUCs) using the Hanley–McNeil method.10 The cut-off points were calculated with MedCalc software as the points with the best sensitivity–specificity balance. Sensitivities at 85 and 95% (15 and 5% false-positive rate respectively) fixed specificities, and positive and negative likelihood ratios (LRs) were also calculated.

Results

Table 1 shows the clinical characteristics of the groups of both populations enrolled in the study. The teaching set consisted of 54 normal eyes and 73 glaucomatous eyes (58 with primary open-angle glaucoma, 13 with pseudoexfoliative glaucoma, and 2 with pigmentary glaucoma). The mean age was 59.1±10.4 years for the normal group and 61.9±7.0 years for the glaucoma group. The validating set comprised 70 normal eyes and 67 glaucoma eyes (57 with primary open-angle glaucoma, 9 with pseudoexfoliative glaucoma, and 1 with pigmentary glaucoma). The mean age of the normal group was 57.2±10.9 years and the mean age of the glaucomatous group was 59.1±8.3 years. Age and central corneal thickness did not differ significantly between the groups in either sample.

Table 1 Clinical characteristics of both populations included in the study
Full size table

Table 2 shows the mean values and standard deviation of all parameters evaluated in the teaching and validating sets. The values of all studied OCT parameters, except disc area, were different between the normal and glaucoma groups in both populations (P<0.001).

Table 2 Mean and SD values of optic nerve head parameters of optical coherence tomography in the teaching and validating sets
Full size table

A stepwise procedure was used to identify the ONH parameter of OCT that accounted for the greatest amount of error, which was then included into the model, then the next best variable was identified and included, and so on. The horizontal rim width was selected at the first iteration, and the disc area was selected at the second iteration. Using this procedure, our LDF was defined as follows: LDF=8.204+(2.59 × disc area)–(9.25 × horizontal rim width).

In the teaching set, the highest sensitivity–specificity balance was observed for our LDF (76.7–100%; cut-off point: >1.26) and vertical rim area (76.7–94.6%; cut-off point: ⩽0.17). The AUC was 0.941 (95% confidence interval (CI): 0.885–0.974) for the LDF. The largest AUCs for the provided OCT parameters were 0.917 (95% CI: 0.855–0.958) for the vertical rim area, 0.915 (95% CI: 0.853–0.957) for the horizontal rim width, and 0.911 (95% CI: 0.848–0.954) for the cup/disc vertical ratio. There were no differences between the AUCs of these parameters, except between those of the LDF and the horizontal rim width (P=0.043).

In the validating set (Table 3), the cup/disc area ratio, the cup/disc vertical ratio, and the LDF had the best sensitivity–specificity pairs. The horizontal rim width and the cup/disc area ratio had the highest positive LRs (8.71 and 7.31 respectively), and the LDF and cup/disc area ratio had the lowest negative LRs (0.19 for both). The greatest AUCs (Table 3; Figure 1) were 0.923 (95% CI: 0.864–0.961) for the LDF, 0.898 (95% CI: 0.835–0.943) for the vertical rim area, and 0.886 (95% CI: 0.820–0.934) for the cup/disc vertical ratio. Compared with the OCT-provided parameters (Table 4), the LDF had the largest AUC, except for the vertical rim area (P=0.109) and the cup/disc vertical ratio (P=0.093).

Table 3 In the validating set, areas under the ROC curves, best sensitivity–specificity balance, and likelihood ratios of optic nerve head parameters of OCT to discriminate between normal and glaucoma subjects
Full size table
Figure 1
figure 1

Receiver operating characteristic (ROC) curves of the linear discriminant function (LDF) vertical rim area, and cup/disc vertical ratio between healthy eyes and glaucomatous patients in the validating set. These parameters exhibited the largest areas under the ROC curve (AUC): 0.923 (95% confidence interval: 0.864–0.961), 0.898 (95% confidence interval: 0.835–0.943), and 0.886 (95% confidence interval: 0.820–0.934) respectively. There were no significant differences between them.

Full size image
Table 4 In the validating set, differences between the areas under the ROC curves (Hanley–McNeil method) for optic nerve head parameters of OCT
Full size table

The LDF, cup/disc vertical ratio, and cup/disc area ratio yielded sensitivities of 83.6% at a fixed specificity of 85%, whereas the sensitivities were 71.6, 64.2, and 61.2%, respectively, at a fixed specificity of 95%.

Discussion

Previous studies11, 12, 13, 14, 15, 16 have reported the sensitivity and specificity of OCT for discriminating between healthy and glaucomatous eyes. Very few studies,17, 18, 19, 20, 21 however, have attempted to increase the diagnostic ability of OCT using learning classifiers. In most of these studies, RNFL parameters,20 RNFL and ONH variables,17, 18, 19 or merged RNFL and SAP parameters were combined.21 We were unable to find other published studies aimed at calculating an LDF based solely on ONH parameters measured with OCT.

In the present study, disc areas were similar between the normal and glaucoma groups. This is a critical factor when comparing groups because most ONH parameters are directly related to disc size.22 Moreover, the optic disc morphology correlates with the evaluated test (ONH parameters of OCT); therefore, here we classified healthy and glaucomatous eyes regardless of optic disc appearance to avoid overestimating the sensitivity and specificity of OCT.23, 24 By ignoring the ONH and RNFL appearance, we might have included pre-perimetric glaucoma patients in the normal group, which would lead to an underestimation of the true specificity. This is a limitation that is inherent in the design of any study like this.

The Stratus OCT software automatically determines the disc margin as the end of the retinal pigment epithelium/choriocapillaris layer and interpolates between the scans to obtain measurements throughout the ONH.4 Although the Stratus OCT partially infers ONH measurements, in our study most ONH parameters showed a good ability to discriminate between healthy and glaucoma patients with perimetric glaucoma. The LDF yielded the highest sensitivities at high specificities compared to any one single ONH parameter determined using OCT. Depending on the pre-test probability, positive or negative LRs indicate the extent to which a factor (ie, the probability of disease) will increase or decrease respectively. An LR value close to 1 indicates insignificant effects, whereas LR values higher than 10 or lower than 0.1 often indicate higher post-test odds of the disease. The LDF had the lowest negative LR, thus normal results were associated with a high post-test probability of disease for these variables, indicating a better ability to exclude the presence of glaucoma. This is the strength of the LDF, which yielded a good sensitivity (72%) at 95% specificity. An LR value higher than 0.013 for the LDF (cut-off point for 95% specificity) virtually rules out the chance that the patient has the disease.

Huang and Chen17 and Chen et al18 compared automated classifications for glaucoma and developed a logistic regression analysis including both RNFL thickness and ONH parameters obtained with the OCT. Medeiros et al19 also calculated an LDF and validated it in an independent population, but the size of the validation sample was relatively small and contained a high proportion of moderate and advanced cases. Huang and Chen17 and Chen et al18 reported AUCs lower than 0.75 for the ONH parameters. In their study, the cup/disc vertical ratio was the parameter with the best sensitivity–specificity balance. Our results are consistent with those of Medeiros et al19 in which AUCs were higher than 0.8 for all ONH parameters, except disc area. These studies17, 18, 19 included more parameters in their analyses and all of them used normal optic disc morphology to classify the normal group. Thus, their functions required two scan protocols and two analysis protocols, which potentially introduces an additional source of variability, increases the time required to perform the test, extends the time needed to interpret the results (two analyses and more variables in the equation), and increases the cost per examination. The formula of our LDF was based on only two ONH parameters and yielded a very good sensitivity–specificity balance in the validation sample for glaucoma diagnosis.

Different designs and severity of visual field defects make it difficult to compare results among diverse studies. Obviously, the severity of visual field loss has an important effect on imaging instrument sensitivity.22 More severe disease is associated with increased sensitivity; therefore, in populations of patients with more advanced visual field losses, a better diagnostic accuracy for the LDF and most ONH parameters might be expected. In our study, the results in the validating set confirmed those obtained in the teaching set. Nevertheless, the sensitivity–specificity balance of the LDF and other ONH parameters was somewhat lower in the validating set. The validating set, however, comprised glaucoma eyes with milder visual field defects, which may explain the lower diagnostic ability of OCT in this population.

ONH appearance varies widely among healthy subjects, limiting the usefulness of the LDF and single parameters to differentiate between glaucoma and normal subjects, particularly in small and large optic discs. The age and ethnic characteristics of the validation sample were similar to those of the teaching set, and this may have biased the findings towards our LDF when compared to other OCT parameters in the second population. Only glaucoma patients with high IOP were included, and therefore our results may not be applicable to subjects with low-tension glaucoma. Also, the diagnostic ability of this LDF might be lower for pre-perimetric glaucoma, because it was designed to detect glaucoma patients with visual field losses.

The lower the image quality, the lower the accuracy of the OCT measurements. The quality of the data obtained by the imaging devices is influenced by media opacity, retinal pigment epithelium status, instrument variability, and positioning and centring of the images. In our study, we selected only good quality scans, but in clinical practice this is not always possible. All these limitations must be taken into account when interpreting OCT results.

Although we were unable to find other combinations of ONH parameters with better diagnostic ability, other statistical analyses could provide alternative formulas that would increase the diagnostic performance of OCT.

The AUC of the proposed LDF was not significantly different from the vertical rim area and the cup/disc vertical ratio, but still the sensitivity values were higher for the LDF at a high fixed-specificity. Also, the lowest negative LR was found for the LDF (and for the cup/disc area ratio), and these characteristics give the LDF an advantage for classifying healthy subjects as normal subjects. This is a key point when clinicians must discriminate between normal and early glaucoma patients. Hence, further studies with this LDF and other discriminant functions are needed to determine the ability of learning classifiers at early stages of the disease or in certain circumstances, such as recognizing glaucoma when it is difficult to decide on the basis of other parameters. Linear classifiers are additional tools and should be used in combination with the rest of the parameters and clinical explorations.