Acessibilidade / Reportar erro

Low specificity and sensitivity of smell identification testing for the diagnosis of Parkinson’s disease

Baixa sensibilidade e especificidade do teste de identificação de odores para o diagnóstico da doença de Parkinson

Abstracts

Objective:

The aim of this study is to determine if the University of Pennsylvania’s Smell Identification Test (UPSIT) is an accurate diagnostic tool for olfactory dysfunction in Parkinson’s disease (PD).

Method:

We included 138 non-demented PD subjects and 175 control subjects matched by gender. Smell identification was tested using UPSIT.

Results:

The mean number of UPSIT items correctly identified by controls was 27.52±5.88; the mean score for PD subjects was 19.66±6.08 (p=<0.001). UPSIT sensitivity was 79.7% with a specificity of 68.5% using a cut-off score of ≤25. The overall accuracy for the diagnosis of PD was of 75.3%.

Conclusion:

UPSIT accuracy and specificity were lower than what has been previously reported. Our data demonstrates that 17.5% of items of the UPSIT were not well identified by healthy controls. Further research of the identification of a truly cross-cultural test is warranted.

Parkinson’s disease; University of Pennsylvania Smell Identification Test; olfaction; diagnostic test


Objetivo:

O objetivo deste estudo é determinar se o University of Pennsylvania Smell Identification Test (UPSIT) é uma ferramenta diagnóstica útil para a caracterizar disfunção olfativa na doença de Parkinson (DP).

Método:

Foram incluídos 138 indivíduos não dementes assuntos PD e 175 indivíduos controle pareados por sexo. Identificação cheiro foi testada usando UPSIT.

Resultados:

O número médio de itens UPSIT corretamente identificados pelos controles foi de 27,52±5,88; para sujeitos com DP foi de 19,66±6,08 (p=<0,001). A sensibilidade do UPSIT foi de 79,7%, com especificidade de 68,5%, utilizando um ponto de corte de ≤25. A exatidão global para o diagnóstico de DP foi de 75,3%.

Conclusão:

A precisão e a especificidade do UPSIT foram menores do que o que foi relatado anteriormente. Nossos dados demonstram que 17,5% dos itens da UPSIT não foram adequadamente identificados pelos controles saudáveis. São necessárias outras pesquisas para a identificação de um teste verdadeiramente cross-cultural nessa área.

doença de Parkinson; University of Pennsylvania Smell Identification Test; olfato; teste de diagnostic


Currently, the diagnosis of Parkinson’s disease (PD) relies mainly on clinical assessment and criteria. Dopamine transporter scans are useful for the differential diagnosis of some parkinsonisms 1. Brajkovic LD, Svetel MV, Kostic VS, et al. Dopamine transporter imaging (123)i-fp-cit (datscan) spet in differential diagnosis of dopa-responsive dystonia and young-onset Parkinson’s disease. Hell J Nucl Med 2012;15:134-138. , 2. Bajaj N, Hauser RA, Grachev ID. Clinical utility of dopamine transporter single photon emission ct (dat-spect) with (123i) ioflupane in diagnosis of parkinsonian syndromes. J Neurol Neurosurg Psychiatry 2013; Epub Ahead if Print. ; nevertheless, their availability in developing countries is limited.

In recent years, the diagnosis of non-motor symptoms of PD has acquired greater clinical relevance. These symptoms may be present early in the disease and, as such, may prove to be useful as potential biomarkers for premotor diagnosis 3. Chaudhuri KR, Healy DG, Schapira AH. Non-motor symptoms of Parkinson’s disease: diagnosis and management. Lancet Neurol 2006;5:235-245 .

Olfactory dysfunction is one of the most prevalent non-motor symptoms of PD. It is found in up to 90% of subjects 4. Hawkes CH, Shephard BC. Selective anosmia in Parkinson’s disease? Lancet 1993;341:435-436. . Furthermore, hyposmia may be detected years before motor symptomatology develops 5. Haehner A, Mayer AM, Landis BN, et al. High test-retest reliability of the extended version of the “sniffin’ sticks” test. Chem Senses 2009;34:705-711.

. Ponsen MM, Stoffers D, Booij J, van Eck-Smit BL, Wolters E, Berendse HW. Idiopathic hyposmia as a preclinical sign of Parkinson’s disease. Ann Neurol 2004;56:173-181.
- 7. Ross GW, Petrovitch H, Abbott RD, et al. Association of olfactory dysfunction with risk for future Parkinson’s disease. Ann Neurol 2008;63:167-173. . The Braak hypothesis proposes the olfactory bulb as one of the regions affected in early stages of PD 8. Braak H, Ghebremedhin E, Rub U, Bratzke H, Del Tredici K. Stages in the development of Parkinson’s disease-related pathology. Cell Tissue Res 2004;318:121-134. .

Among olfactory testing, smell identification tests are the most frequently used in PD subjects, mainly because they are easy to apply and interpret 9. Silveira-Moriyama L, Carvalho Mde J, Katzenschlager R, et al. The use of smell identification tests in the diagnosis of Parkinson’s disease in Brazil. Mov Disord 2008;23:2328-2334. . These tests have been proposed as useful markers for the detection of PD in first-degree relatives of subjects with hereditary PD 6. Ponsen MM, Stoffers D, Booij J, van Eck-Smit BL, Wolters E, Berendse HW. Idiopathic hyposmia as a preclinical sign of Parkinson’s disease. Ann Neurol 2004;56:173-181. , 7. Ross GW, Petrovitch H, Abbott RD, et al. Association of olfactory dysfunction with risk for future Parkinson’s disease. Ann Neurol 2008;63:167-173. , 1010 . Ponsen MM, Stoffers D, Wolters E, Booij J, Berendse HW. Olfactory testing combined with dopamine transporter imaging as a method to detect prodromal Parkinson’s disease. J Neurol Neurosurg Psychiatry 2010;81:396-399. . Moreover, smell identification tests may be valuable to differentiate PD from other movement disorders 1111 . Ondo WG, Lai D. Olfaction testing in patients with tremor-dominant Parkinson’s disease: is this a distinct condition? Mov Disord 2005;20:471-475. .

The only published account of formal olfactory testing in Mexican PD subjects was carried out using the brief smell identification test (BSIT) 1212 . Rodriguez-Violante M, Lees AJ, Cervantes-Arriaga A, Corona T, Silveira-Moriyama L. Use of smell test identification in Parkinson’s disease in Mexico: a matched case-control study. Mov Disord 2011;26:173-176. . BSIT is a shorter version of the University of Pennsylvania Smell Identification Test (UPSIT), and, as such, it is quicker to apply. This test showed a 71.4% sensitivity and 85.7% specificity, with an accuracy of 78.6% to differentiate PD subjects from healthy controls. Because of the greater number of items; the 40-item UPSIT may have a greater accuracy than the BSIT, for the identification of PD subjects.

The aim of this study is to determine if UPSIT is a more accurate diagnostic tool for PD in Mexican population.

METHODS

We recruited 138 non-demented (Montreal Cognitive Assessment Test 1313 . Chou KL, Amick MM, Brandt J, et al. A recommended scale for cognitive screening in clinical trials of Parkinson’s disease. Mov Disord 2010;25:2501-2507. score 26) subjects, who fulfilled the Queen Square Brain Bank Criteria 1414 . Gibb WR, Lees AJ. The relevance of the lewy body to the pathogenesis of idiopathic Parkinson’s disease. J Neurol Neurosurg Psychiatry 1988;51:745-752. for PD, from the movement disorder specialist clinic at the National Institute of Neurology and Neurosurgery, in Mexico City, Mexico. None of the subjects had undergone functional neurosurgery for PD. Also, 175 healthy controls were recruited among visitors to the hospital. Exclusion criteria for both groups included history of seasonal allergies, nasal illness or surgery, severe head trauma or current upper respiratory tract infection. Control subjects with a first or second-degree family history of any neurodegenerative disease including PD were also excluded.

We collected sociodemographic variables including gender, age, smoking status and place of residence (urban or rural). PD severity was determined according to the Hoehn and Yahr (HY) scale 1515 . Hoehn MM, Yahr MD. Parkinsonism: Onset, progression and mortality. Neurology 1967;17: 427-442. .

The local research and ethics committee approved this study. All participants provided written informed consent.

Smell testing

The Spanish version of the University of Pennsylvania Smell Identification Test (UPSIT, Sensonics, Haddon Heights, New Jersey, USA) was used. The UPSIT consists of four booklets, each containing ten microencapsulated odors. The test was applied by scratching each microcapsule with a pencil tip and presented immediately to the subject for sniffing. The subject was then offered four possible responses for each odor and was asked to select one of the options even if no smell was perceived or identified.

Statistical analysis

Demographic data was reported in terms of percentages, mean and standard deviation. The percentage of items correctly identified by control subjects and PD subjects was compared using a χ2 test. The UPSIT total score between groups was compared by an independent t-student test. To evaluate the factors that independently affected UPSIT score, we performed multiple linear regressions. UPSIT sensitivity and specificity estimates were determined using a Receiver Operator characteristic (ROC) curve analysis. A PD probability curve (estimated probability of belonging to the PD group) was determined by logistic regression with PD as outcome and the UPSIT score as independent variable.

RESULTS

Demographic data for both groups is shown in Table . There was no significant difference in age (p=0.233) or gender (p=0.087) between control and PD subjects. Mean disease duration for PD subjects was 7.4±6.1 years. In regards to disease stage, 73.2% of PD subjects presented at a mild stage of disease (HY 1-2), 21.7% presented a moderate stage (HY 3) and 5.1% presented a severe stage (HY 4-5).

Table
. Comparison of demographic data.

Smell identification in PD and control subjects

The mean score for controls was 27.52±5.88; mean score for PD subjects was 19.66±6.08 (p=<0.001). The distribution of correctly identified items by each group is shown on Figure 1 . Identification of the items cheese, peach, pickles, pine and soap was not significantly different between control and PD subjects.

Figure 1
. Number of subjects who correctly identified each item in UPSIT.

Multiple regression analysis

Multiple linear regressions showed diagnosis (p=<0.001, B=7.48), age (p=<0.001, B=-0.262) and gender (p=0.001, B=2.505) to be significant independent predictors of UPSIT score. Neither area of residence (p=0.71) or smoking status (p=0.9) were significant.

UPSIT sensitivity and specificity

UPSIT sensitivity was 79.7% with a specificity of 68.5% using a cut-off score ≤25 (Area under the curve =0.82, IC 95% 0.77-0.87, p=<0.001). The overall accuracy of the diagnosis of PD was 75.3%. The ROC curve is presented as Figure 2 . Since age and gender were significant determinants of the UPSIT total score for both groups, we divided subjects into women more than 67 years old and women with an age of 67 or less. In this case, UPSIT sensitivity for PD detection was 85.7% with a 62.6% specificity for a cutoff score ≤25 (Area under the curve =0.79, IC 95% 0.74-0.84, p=<0.001). PD probability related to the UPSIT total score is presented as Figure 3 .

Figure 2
. Sensitivity and specificity of UPSIT.

Figure 3
. Parkinson’s disease probability curve for the UPSIT.

DISCUSSION

This is the first study to report UPSIT testing in Mexican PD subjects. UPSIT was more sensitive than B-SIT for Mexican population. Nevertheless, both specificity and accuracy with UPSIT were lower than those reported using the B-SIT. In addition, UPSIT specificity for the detection of PD is lower than expected based on other international reports. Optimal cut-off values are also lower than those proposed by previous studies 1616 . Doty RL, Bromley SM, Stern MB. Olfactory testing as an aid in the diagnosis of Parkinson’s disease: development of optimal discrimination criteria. Neurodegeneration 1995;4:93-97. , 1717 . Dotchin C, Msuya O, Walker R. The use of smell identification tests in the diagnosis of Parkinson’s disease in Brazil. Mov Disord 2010;25:510-511. . In a previous study, the UPSIT total score was significantly lower in healthy subjects living in Mexico City, when compared to healthy subjects living in areas with milder air pollution 1818 . Calderon-Garciduenas L, Torres-Jardon R. Air pollution, socioeconomic status, and children’s cognition in megacities: the Mexico city scenario. Front Psychol 2012;3:217. . Eighty-three percent of subjects in our study lived in an urban area (Mexico City); as such, they are exposed to severe air pollution, which may have impaired total UPSIT scores, particularly in control subjects.

Of the 40 items included in UPSIT, seven were identified by less than 50% of control subjects. These items were rose, cherry, nut, turpentine, pizza, cheese and pickles. In the United States, most of the UPSIT items are correctly identified by more than 90% of healthy subjects 1919 . Doty RL, Marcus A, Lee WW. Development of the 12-item cross-cultural smell identification test (cc-sit). Laryngoscope 1996;106:353-356. . This suggests that some items may not be adequate to evaluate olfactory dysfunction in a Mexican population, probably due to cultural issues. Silveira-Moriyama et. al. also reported a decreased identification of the odors pizza, cheese and pickles in Brazilian control subjects 9. Silveira-Moriyama L, Carvalho Mde J, Katzenschlager R, et al. The use of smell identification tests in the diagnosis of Parkinson’s disease in Brazil. Mov Disord 2008;23:2328-2334. . In Japan, ten of the UPSIT odorants, including cherry, clove, fruit punch, licorice, lilac, lemon, paint thinner and pine, had an identification rate of less than 70% in normal subjects 2020 . Kondo H, Matsuda T, Hashiba M, Baba S. A study of the relationship between the t&t olfactometer and the university of pennsylvania smell identification test in a Japanese population. Am J Rhinol 1998;12:353-358. .

UPSIT has been used in previous studies for the detection of PD. Doty et al report sensitivity and specificity between 76% and 91% according to gender and age for PD detection 1616 . Doty RL, Bromley SM, Stern MB. Olfactory testing as an aid in the diagnosis of Parkinson’s disease: development of optimal discrimination criteria. Neurodegeneration 1995;4:93-97. . In Brazil, a sensitivity of 82.1% with a specificity of 83.5% for the diagnosis of PD was reported 9. Silveira-Moriyama L, Carvalho Mde J, Katzenschlager R, et al. The use of smell identification tests in the diagnosis of Parkinson’s disease in Brazil. Mov Disord 2008;23:2328-2334. . In Mexico, the only published account of olfactory testing in PD subjects was carried out using B-SIT. The accuracy of this test for Mexican PD subjects was lower than what had been reported for other developing countries 1212 . Rodriguez-Violante M, Lees AJ, Cervantes-Arriaga A, Corona T, Silveira-Moriyama L. Use of smell test identification in Parkinson’s disease in Mexico: a matched case-control study. Mov Disord 2011;26:173-176. .

Regarding other smell identification tests, the SS-16 in Brazilian population had a specificity of 89% with a sensitivity of 81.1% 9. Silveira-Moriyama L, Carvalho Mde J, Katzenschlager R, et al. The use of smell identification tests in the diagnosis of Parkinson’s disease in Brazil. Mov Disord 2008;23:2328-2334. . Another study on 70 Brazilian patients and 70 controls showed that the SS-16 had a sensibility of 88.2% and specificity of 85.7% for discriminating late onset PD from controls. Sensitivity and specificity was significantly lower (63.2% and 78.6% respectively) when trying to establish a diagnosis of early onset Parkinson’s disease 2121 . Santin R, Fonseca VF, Bleil CB, Rieder CR, Hilbig A. Olfactory function and Parkinson’s disease in Southern Brazil. Arq Neuropsiquiatr 2010;68:252-257. . A small study in Chilean PD patients and controls, with 40 subjects in each group, reported a sensitivity of 100% and a specificity of 95% on the SS-12 2222 . Miranda M, Slachevsky A, Perez C. [preliminary validation of a smelling test in a sample of chilean population and its role in the differential diagnosis of Parkinson’s disease]. Rev Med Chil 2006;134:1071-1072. .

Most international studies report age, gender and smoking status as factors that affect UPSIT total score. In the present study, subjects that were male or older corresponded to a lower UPSIT total score. Smoking has been related with a low risk of PD thus a difference on smoking status between PD subjects and control subjects was expected 2323 . Fratiglioni L, Wang HX. Smoking and Parkinson’s and Alzheimer’s disease: review of the epidemiological studies. Behav Brain Res 2000;113:117-120. . The lack of that difference in our study may reflect a selection bias due to the fact that controls were recruited from visitors to the hospital. Nevertheless, the homogeneity between groups reduces the risk of a non-random bias.

Limitations of this study include the fact that PD diagnosis was done solely on clinical evaluation, as confirmation of PD either by functional assessment or histopathology was not performed. Because of this, it is possible that some parkinsonisms may have been included in the PD group.

It was carried out in subjects from a reference center, which may limit result generalization to primary care.

Also, most subjects included in this study lived in an area with high levels of air pollution, which may affect olfaction. Therefore, future studies involving control subjects matched by area of residence may be needed.

Finally, although a Spanish version of the UPSIT was used, some of the odors of this test may not be familiar to a Mexican population. Our data demonstrates that 17.5% of items of the UPSIT were not well identified by healthy controls. Given the increasing importance of smell testing as a diagnostic tool for Parkinson’s disease and other movement disorders, further research for the identification of either a truly cross-cultural test or a population-specific test is warranted.

References

  • 1
    Brajkovic LD, Svetel MV, Kostic VS, et al. Dopamine transporter imaging (123)i-fp-cit (datscan) spet in differential diagnosis of dopa-responsive dystonia and young-onset Parkinson’s disease. Hell J Nucl Med 2012;15:134-138.
  • 2
    Bajaj N, Hauser RA, Grachev ID. Clinical utility of dopamine transporter single photon emission ct (dat-spect) with (123i) ioflupane in diagnosis of parkinsonian syndromes. J Neurol Neurosurg Psychiatry 2013; Epub Ahead if Print.
  • 3
    Chaudhuri KR, Healy DG, Schapira AH. Non-motor symptoms of Parkinson’s disease: diagnosis and management. Lancet Neurol 2006;5:235-245
  • 4
    Hawkes CH, Shephard BC. Selective anosmia in Parkinson’s disease? Lancet 1993;341:435-436.
  • 5
    Haehner A, Mayer AM, Landis BN, et al. High test-retest reliability of the extended version of the “sniffin’ sticks” test. Chem Senses 2009;34:705-711.
  • 6
    Ponsen MM, Stoffers D, Booij J, van Eck-Smit BL, Wolters E, Berendse HW. Idiopathic hyposmia as a preclinical sign of Parkinson’s disease. Ann Neurol 2004;56:173-181.
  • 7
    Ross GW, Petrovitch H, Abbott RD, et al. Association of olfactory dysfunction with risk for future Parkinson’s disease. Ann Neurol 2008;63:167-173.
  • 8
    Braak H, Ghebremedhin E, Rub U, Bratzke H, Del Tredici K. Stages in the development of Parkinson’s disease-related pathology. Cell Tissue Res 2004;318:121-134.
  • 9
    Silveira-Moriyama L, Carvalho Mde J, Katzenschlager R, et al. The use of smell identification tests in the diagnosis of Parkinson’s disease in Brazil. Mov Disord 2008;23:2328-2334.
  • 10
    Ponsen MM, Stoffers D, Wolters E, Booij J, Berendse HW. Olfactory testing combined with dopamine transporter imaging as a method to detect prodromal Parkinson’s disease. J Neurol Neurosurg Psychiatry 2010;81:396-399.
  • 11
    Ondo WG, Lai D. Olfaction testing in patients with tremor-dominant Parkinson’s disease: is this a distinct condition? Mov Disord 2005;20:471-475.
  • 12
    Rodriguez-Violante M, Lees AJ, Cervantes-Arriaga A, Corona T, Silveira-Moriyama L. Use of smell test identification in Parkinson’s disease in Mexico: a matched case-control study. Mov Disord 2011;26:173-176.
  • 13
    Chou KL, Amick MM, Brandt J, et al. A recommended scale for cognitive screening in clinical trials of Parkinson’s disease. Mov Disord 2010;25:2501-2507.
  • 14
    Gibb WR, Lees AJ. The relevance of the lewy body to the pathogenesis of idiopathic Parkinson’s disease. J Neurol Neurosurg Psychiatry 1988;51:745-752.
  • 15
    Hoehn MM, Yahr MD. Parkinsonism: Onset, progression and mortality. Neurology 1967;17: 427-442.
  • 16
    Doty RL, Bromley SM, Stern MB. Olfactory testing as an aid in the diagnosis of Parkinson’s disease: development of optimal discrimination criteria. Neurodegeneration 1995;4:93-97.
  • 17
    Dotchin C, Msuya O, Walker R. The use of smell identification tests in the diagnosis of Parkinson’s disease in Brazil. Mov Disord 2010;25:510-511.
  • 18
    Calderon-Garciduenas L, Torres-Jardon R. Air pollution, socioeconomic status, and children’s cognition in megacities: the Mexico city scenario. Front Psychol 2012;3:217.
  • 19
    Doty RL, Marcus A, Lee WW. Development of the 12-item cross-cultural smell identification test (cc-sit). Laryngoscope 1996;106:353-356.
  • 20
    Kondo H, Matsuda T, Hashiba M, Baba S. A study of the relationship between the t&t olfactometer and the university of pennsylvania smell identification test in a Japanese population. Am J Rhinol 1998;12:353-358.
  • 21
    Santin R, Fonseca VF, Bleil CB, Rieder CR, Hilbig A. Olfactory function and Parkinson’s disease in Southern Brazil. Arq Neuropsiquiatr 2010;68:252-257.
  • 22
    Miranda M, Slachevsky A, Perez C. [preliminary validation of a smelling test in a sample of chilean population and its role in the differential diagnosis of Parkinson’s disease]. Rev Med Chil 2006;134:1071-1072.
  • 23
    Fratiglioni L, Wang HX. Smoking and Parkinson’s and Alzheimer’s disease: review of the epidemiological studies. Behav Brain Res 2000;113:117-120.

Publication Dates

  • Publication in this collection
    Jan 2014

History

  • Received
    17 May 2013
  • Reviewed
    23 July 2013
  • Accepted
    30 July 2013
Academia Brasileira de Neurologia - ABNEURO R. Vergueiro, 1353 sl.1404 - Ed. Top Towers Offices Torre Norte, 04101-000 São Paulo SP Brazil, Tel.: +55 11 5084-9463 | +55 11 5083-3876 - São Paulo - SP - Brazil
E-mail: revista.arquivos@abneuro.org