Language and cultural shortcomings of current cognitive tests

Cognitive tests, which are incorporated in both the screening and diagnosis of dementia and MCI, allow for early detection and accuracy in neuropsychiatric assessment.^{Reference Panegyres, Berry and Burchell16} However, the majority of cognitive tests available at present are designed for English speakers in the context of Western cultures such as those in Europe and North America.^{Reference Tanveer, Croucher and Porter1,Reference Tuerk and Sauer17,Reference Fillenbaum, Heyman, Williams, Prosnitz and Burchett18} They were standardised on male, White out-patients, not accounting for the populations of non-Western countries, including LMICs.^{Reference Tuerk and Sauer17,Reference Fillenbaum, Heyman, Williams, Prosnitz and Burchett18}

Owing to the content of cognitive tests, both language and culture affects an individual's perception of test questions and responses to them. When these English-language cognitive tests are administered to non-English speakers and to those from Western cultural backgrounds, such as Europeans and North Americans, we see reduced sensitivity and specificity and higher rates of false-positive and false-negative scores for the non-native English speakers.^{Reference Regan19–Reference Khan and Tadros22} Thus, there is a gap in the tests available to assess the cognitive health of culturally diverse populations, as well as populations in countries in the global East.

One proposed solution is to develop and pilot new cognitive tests but this is a time-consuming endeavour and not always feasible.^{Reference Beaton, Bombardier, Guillemin and Ferraz23} An alternative is to adjust cut-off scores of existing cognitive tests for different language-speaking groups and cultural populations but this is criticised for reducing sensitivity, specificity and likelihood ratios.^{Reference Bohnstedt, Fox and Kohatsu24,Reference Milani, Marsiske, Cottler, Chen and Striley25}

A third solution in practice has been to translate existing cognitive tests into target languages, with the use of the back-translation method.^{Reference Yu, Lee and Woo26} This involves translating a test directly into a target language, translating it back into the original language and comparing the old and new versions to see if there are any discrepancies.^{Reference Yu, Lee and Woo26} However, despite the rate at which back-translation is used, it still results in cognitive tests that produce false-positive and false-negative scores.^{Reference Wang, Lee and Fetzer27} This is because the process of changing a cognitive test from one language to another and then back-translating can result in words being mistranslated, questions developing different meanings and loss of the concepts being assessed.^{Reference Yu, Lee and Woo26–Reference Flaherty, Gaviria, Pathak, Mitchell, Wintrob and Richman28}

Furthermore, translation does not account for cross-cultural influence on the ability to answer questions beyond fluency in a target language.^{Reference Tanveer, Croucher and Porter1,Reference Wang, Lee and Fetzer27,Reference Flaherty, Gaviria, Pathak, Mitchell, Wintrob and Richman28} This is commonly seen in cognitive tests as they utilise questions requiring familiarity with the Western alphabet, names, idioms or history to assess cognitive domains such as language and memory.^{Reference Parker and Philp21,Reference Wang, Lee and Fetzer27,Reference Flaherty, Gaviria, Pathak, Mitchell, Wintrob and Richman28} This cultural bias in test questions compromises the reliability of these cognitive tests:^{Reference Van de Vijver and Tanzer29,Reference Vijver30} the same person may produce different scores owing to misunderstandings propagated by mismatched language and culture.^{Reference Wang, Lee and Fetzer27} The validity of these cognitive tests is also questionable: we are uncertain whether the results are due to the individual's state of health or due to errors in translation or differences in culture.^{Reference Wang, Lee and Fetzer27} Therefore, existing cognitive tests require not just translation but adaptation for different cultures.^{Reference Van de Vijver and Tanzer29–Reference Guillemin, Bombardier and Beaton31}

A model for the translation and cultural adaptation of cognitive tests

Flaherty et al^{Reference Flaherty, Gaviria, Pathak, Mitchell, Wintrob and Richman28} proposed that the first step in translating and culturally adapting any health measure is the selection process: literature is searched to see which tests may be suitable for a target population.^{Reference Flaherty, Gaviria, Pathak, Mitchell, Wintrob and Richman28,Reference Waheed32} A list of potential tests is drawn up and one test is hand-selected based on whether it has been used in cultural groups similar to the target population, whether it has been translated for several languages before and whether it has been psychometrically validated for diverse groups.^{Reference Flaherty, Gaviria, Pathak, Mitchell, Wintrob and Richman28,Reference Waheed32} There are also considerations beyond language and culture, such as the frequency of a test's use across published literature, the format it is delivered in and the cost of use due to copyright. Once a test has been selected according to this guidance it is determined whether it is available for the target population's language and cultural context. Owing to the need for cross-cultural health measures, if the selected test is not available in both the target language and cultural context then it requires translation and cultural adaptation. There may be instances where an adapted version of a test matches the target language but not the cultural context, or vice versa. In that instance further translating or culturally adapting that version of the test is appropriate as an attempt at adaptation has already been started.

As regards cognitive testing, our team demonstrated that, for every cognitive test, incorporation of literature on that test and feedback from adaptors of that test allows the development of a set of guidelines on culturally adapting each of that test's questions.^{Reference Waheed, Mirza, Waheed, Malik and Panagioti33} These methods were illustrated through forming cultural adaptation guidelines for the Addenbrooke's Cognitive Examination Version III^{Reference Waheed, Mirza, Waheed, Malik and Panagioti33} and we later replicated this for the Montreal Cognitive Assessment.^{Reference Khan, Mirza and Waheed34}

Following this methodology, a cognitive test can be translated and culturally adapted for any language and cultural context using its guidelines.^{Reference Waheed, Mirza, Waheed, Malik and Panagioti33,Reference Khan, Mirza and Waheed34} Such newly adapted tests can then be culturally validated, a process in which they are administered to healthy controls within the target population and a cognitive interviewing approach is undertaken.^{Reference Mirza, Panagioti and Waheed35} This determines whether the questions in the newly adapted version are both understandable and acceptable in those without cognitive impairment and ascertains that there is no bias in the language and cultural context that may cause poor performance.^{Reference Mirza, Panagioti and Waheed35} Finally, the new version of the cognitive test can undergo a psychometric validation in people with dementia or MCI within the target population.

We propose a translation and cultural adaptation model consisting of the above steps, which also incorporates not just selecting and culturally adapting an appropriate test, such as a cognitive test, but also assessing the quality of the adaptation (Fig. 1 outlines the model in the context of cognitive testing).

Fig. 1 Proposed model for the translation and cultural adaptation of health measures in the context of cognitive testing.

When selecting a cognitive test for a target population, finding one that has made accommodations for cultural context alone is not enough; quality assurance should be undertaken to determine whether the translation and cultural adaptation is of good quality. Similarly, if a suitable cognitive test is not available and one has to be translated and culturally adapted, a final quality assurance check can identify whether the translation or cultural adaptation needs to be changed or improved further. We therefore developed the Manchester Translation Evaluation Checklist (MTEC),^{Reference Waheed32} a new tool for assessing the quality of the translation and cultural adaptation of questions in a health measure, to address this.

Aims

In this paper we formally introduce the MTEC and demonstrate its utility as a quality assurance tool for the translation and cultural adaptation of health measures, particularly cognitive tests, by assessing an existing translation of a widely implemented cognitive test, the Mini-Mental State Examination (MMSE). The MMSE is a cognitive test that was developed to screen for both dementia and MCI.^{Reference Folstein, Folstein and McHugh36} It consists of 13 overall questions, some with further subquestions, giving a total of 19 questions. It primarily assesses the cognitive domain of language, but also assesses orientation, registration, attention, memory and visuospatial abilities.^{Reference Larner37}

The MMSE is one of the most commonly used cognitive tests globally. However, when translated and psychometrically validated in several non-English speaking countries, it demonstrated varied sensitivity and specificity owing to cultural biases^{Reference Vas, Pinto, Panikker, Noronha, Deshpande and Kulkarni38–Reference Espino, Lichtenstein, Palmer and Hazuda40} and differing education levels.^{Reference De Yébenes, Otero, Zunzunegui, Rodríguez-Laso, Sánchez-Sánchez and Del Ser41} This is attributed to the heavy reliance on language-based tasks, which often require cultural adaptation beyond literal translation.^{Reference Waheed, Mirza, Waheed, Malik and Panagioti33} Based on our proposed model, use of a quality assurance tool prior to psychometric validation may have highlighted where further translation and cultural adaptation was needed.

As it has universal application in the screening of dementia and MCI and many existing translations,^{Reference Awan, Shahbaz, Akhtar, Ahmad, Iqbal and Ahmed42} the MMSE was an appropriate cognitive test for this demonstration. Given our own language and cultural background (we are all fluent in English and Urdu and are familiar with Pakistani culture), we utilised the MTEC to assess the quality of the translation and cultural adaptation of an Urdu version of the MMSE that had been translated for a Pakistani population.^{Reference Awan, Shahbaz, Akhtar, Ahmad, Iqbal and Ahmed42}