A predictive screening tool to detect diabetic retinopathy or macular edema in primary health care: construction, validation and implementation on a mobile application

Study population

The population study involved diabetic patients followed by the ophthalmological service of the General University Hospital of Elche (Valencian Community, Spain). These patients are referred from the primary health care services, according to the protocol (Generalitat Valenciana: Conselleria de Sanitat, 2006).

Study design and participants

This cross-sectional, observational study was undertaken in a sample of diabetic patients referred to the ophthalmological service of the General University Hospital of Elche by the primary care teams between October 2012 and June 2013 and who were willing to participate. The sampling procedure consisted of randomly selecting one day each week (not always the same day) and recruiting all the diabetic patients who attended that day, by means of linear systematic sampling. A patient was considered to have diabetes if the diagnosis had been made by a physician (ICD9-MC 250.X). Patients were excluded if they had dementia, high myopia or another macular disorder, had had vitreoretinal surgery, a cataract operation during the previous 3 months, had received laser treatment in the macular area or panphotocoagulation, or were taking anti-angiogenic drugs.

Variables and measurements

The main outcome variable was the presence of at least treatable diabetic retinopathy (severe, very severe or proliferating (Ryan et al., 2013)) or diabetic macular edema (DRDME). The diagnosis of these two disorders was made by clinical ophthalmological examination of the retina by indirect ophthalmoscopy and biomicroscopy of the central retina with a Topcon slit lamp, model SL-8Z (Topcon Corporation, Tokyo, Japan) using a 78 diopter lens (78D aspheric lens; Volk Optical Incorporated Company, Mentor, OH, USA) and indirect ophthalmoscopy with a 28D lens by an expert retinal ophthalmologist. Macular edema was defined as the presence of hard exudates or localized retinal thickening within a distance of 500 µm from the fovea, and the degree of diabetic retinopathy was defined according to the ETDRS study (Early Treatment Diabetic Retinopathy Study Research Group, 1991). The secondary variables were type of diabetes, gender, age (years), glycated hemoglobin (HbA1c) (%), foveal thickness (µm) and visual acuity (best corrected). The source of information for data concerning the type of diabetes, hypertension, dyslipidemia, gender, smoking, age and HbA1c was the clinical history. Best corrected visual acuity was obtained using the Snellen scale. The foveal thickness was obtained at the central area by dilating the pupil with a drop of tropicamide and measuring with spectral domain optical coherence tomography (Topcon 3D OCT 2000; Topcon Corporation^®, Itabashi, Tokyo, Japan). The images were acquired by 512 horizontal linear scans and 128 vertical scans, centered on the fixation point making a 6 × 6 square 3D pattern. The mean retinal thickness was calculated automatically by the software of the device. We measured a 6 mm diameter area, centered on the fovea, thereby using for the study the central 1,000 µm area (the central circle).

Sample size

The construction sample involved data from 106 patients, of which 35 had DRDME. To contrast an area under the ROC curve (AUC) different to 0.5, assuming a 95% confidence level and expecting to find an AUC of 0.9, the contrast power was nearly 100%. Using the same parameters in the validation sample (36 patients, 12 with DRDME), we obtained a contrast power of 97.17% (Hanley & McNeil, 1982).

Statistical methods

The quantitative variables are reported as means and standard deviations, and the qualitative variables as absolute and relative frequencies. All the analyses were done with a significance of 5% and the confidence interval was calculated for each relevant parameter. The complete sample was divided into two parts. One part (80%) to construct the predictive model and the other part (20%) to validate the model constructed.

Construction: we used a binary logistic regression model to predict DRDME using the most clinically relevant variables, taking into account that we could only use a maximum number of explanatory variables in the model (10 observations of our outcome for each explanatory variable). The resulting model was transformed into a scoring system using the method of the Framingham study (Sullivan, Massaro & D’Agostino, 2004). The ROC curve was calculated and we constructed groups based on the probabilities of the model: low (<25%), medium (25–50%), high (50–75%) and very high (≥75%).

In the validation sample the AUC was calculated and the observed events compared with the expected events of the model using the X² test. No calculations were made of the classical indicators of a diagnostic test, such as sensitivity, specificity, predictive values and likelihood ratios, as the test constructed does not indicate a single value (positive or negative) but rather a probability of DRDME associated with each score. Accordingly, differences were studied between the expected (given by the predictive model) and the observed events to determine whether the reality corresponded to what was indicated by the model. On the other hand, calculating the AUC indicates the discriminating power of our model. A similar methodology has been used in other studies (Palazón-Bru et al., 2015).

Mobile application

The models were implemented on a mobile application for the operating systems Android and iPhone (see Note S1). This application is free to download from any of the stores. Its name is Diabetic retinopathy predictor.

Ethical consideration

The study was approved by the Ethics Committee of the General Hospital of Elche. All the patients signed the informed consent document. The study was undertaken in accordance with the basic principles of the World Medical Association Declaration of Helsinki and complied with the norms described in the European Union guidelines for good clinical practice.

Strengths and limitations of the study

The main strength of this study concerns its novel way to construct and validate a scoring system to help primary care physicians take decisions about referring a diabetic patient to the ophthalmological specialist. Additionally, the factors used in the model are obtained objectively, so they do not have to be interpreted, as is the case with retinography.

Although the sample size may seem small, it is sufficient for the aims of the study (constructing and validating the model), as the contrast power was greater than 95%, whereas this is generally 80–90%. There are, too, non-significant variables in the model constructed. However, as in other studies, we have to consider that we are evaluating the model overall and not variable by variable, that is the goodness of fit (likelihood ratio test = 53.4, p < 0.001; Nagelkerke R² = 0.583) and the AUC in the validation sample (0.90) (Palazón-Bru et al., 2015). As this model was validated in diabetic patients who had already been referred to the specialist, further studies are needed in the general diabetic population to determine if it is applicable as a screening test for all diabetic patients when they attend their primary health care center (selection bias). These would study the probabilities of DRDME with our scoring system, as the general population would have a low prevalence of this macular disorder. On the other hand, concerning measurement bias, all the measurements were taken using calibrated devices and in accordance with current guidelines. Finally, we did not include other variables in the model that could have influenced DRDME, such as the duration of the disease. This particular variable was not assessed because the clinical history does not record it and the patient knows the duration of the disease only approximately. This could therefore cause an information bias, which is why it was not included in this study. Nevertheless, even without this variable the predictive model had an AUC of 0.90, which equates to a great discriminating power.