Plain English summary

Preference-based measures (PBMs) are surveys that help us understand the quality of life related to health of people or groups. These surveys are essential for making decisions about healthcare approaches and health policies. It is crucial that the PBMs accurately and consistently assess health for a person or group. Currently we lack a summary of information on how PBMs are used specifically for Indigenous people and whether these surveys effectively capture their health experiences. This is particularly important because Indigenous communities worldwide have unique knowledges and approaches to healing that promote their health and care experiences. To address this gap, we reviewed published information to understand how PBMs are used with Indigenous people. Our study shows that PBMs have been used more and more with Indigenous people even though there is not much information on how well these surveys work in this group. This finding highlights the need for further discussions and research on how to measure health-related quality of life accurately and meaningfully for Indigenous people.

Introduction

Preference-based measures provide a measurement of health-related quality of life (HRQL) that can be used to evaluate individual or population health. PBM allow us to calculate value or utility of different health states, which are used in calculating quality-adjusted life years (QALYs) used in economic evaluations. There are two types of PBMs: direct and indirect. Direct PBMs, such as time trade-off or standard gamble, typically ask respondents to make choices about hypothetical health scenarios [1] under conditions of certainty or uncertainty. On the other hand, indirect PBMs use a classification system describing specific dimensions of health, and a scoring system, to infer preferences for a particular health state based on previously elicited values usually from the general population [1]. Indirect generic PBMs (such as the EQ-5D-5L) permit comparison across different populations, whereas indirect condition-specific PBMs (such as the cancer-specific EORTC-QLQ-C30) can be more useful in disease-specific healthcare areas [1].

The usefulness of a PBM is partially contingent on the appropriateness of the PBM for the population in which it is used. Many Indigenous groups hold a worldview and conceptualization of health [2] that may not be reflected in current instruments that have been translated or adapted to non-Indigenous contexts. Indigenous people face distinct health needs and experience significant health inequalities [3,4,5,6], often stemming from the negative impacts of colonial systems [7]. There is a need to improve appropriateness of health care services and measures of disease for Indigenous populations [6]. This means ensuring that PBMs consider the specific cultural contexts, values, and health perspectives of Indigenous communities. It is not only an ethical imperative but also a priority for population health to establish the validity and reliability of PBMs in Indigenous populations.

There is limited evidence on the use of PBMs with Indigenous people, and their alignment with Indigenous worldviews. A systematic review of literature conducted in 2016 [8] identified only one PBM—the EQ-5D 3-level version (EQ-5D-3L) which has been preliminarily validated for use in Māori in New Zealand [9]. Upon conducting a more recent and expanded scoping review (March 2020), we identified an additional seven studies using PBMs with Indigenous people. These studies had various aims including measurement of HRQL, validation of PBMs, and direct valuation of health states [10,11,12,13,14,15,16]. We felt it pertinent to conduct an updated and expanded systematic review on the use of PBMs with Indigenous people, globally.

The objective of this review was to identify, summarize, and appraise the literature reporting the use of direct or indirect PBMs with Indigenous people in terms of (1) assessing measurement properties of PBMs, (2) eliciting health preferences using a direct PBM, (3) reporting development or translation of a PBM for Indigenous Peoples, or (4) measuring HRQL using a PBM.

Methods

Search strategy

Our database search was executed by an expert health librarian (SMC) using the following databases: PROSPERO, OVID Medline, OVID EMBASE, OVID Global Health, OVID Health and Psychosocial Instruments, OVID PsycInfo, Cochrane Library (CDSR and Central), EBSCO CINAHL, EBSCO Econlit, Proquest Dissertations and Theses Global and SCOPUS. The search used controlled vocabulary (e.g., MeSH, Emtree, etc.) and key words representing the concepts “Indigenous people” and “preference-based measures”. The search strategies were informed by the work of Goodwin and Green [17]. Variants of several University of Alberta Health Sciences Search filters were modified for use in each database [18,19,20,21]. No additional limits were applied to the searches. Databases were searched from inception to May 2021, and updated in August 31, 2022. Results (3139) were exported to COVIDENCE review management software. Duplicates (947) were removed. Detailed search strategies are available as an online resource (Supplementary Information).

Screening of records

Records were included in our review if they met the following criteria: (a) the purpose of the study aligned with one of the four categories identified in the objectives; (b) the sample included Indigenous people as the primary population of interest or identified Indigenous people as a specific sub-group; (c) reported the use of any direct (such as time trade-off or standard gamble) or indirect (including generic or condition specific) PBM; (d) were published in English; and (e) were published after inception of PBMs (1980). During the screening process, the reviewers considered the definition of Indigenous people as provided by the authors of each study. If the definition appeared outdated or ambiguous, the screening decisions were guided by the definitions of Indigenous people provided by the United Nations [22, 23] and the Government of Canada [24].

To capture a comprehensive range of literature, reports were retained even if they originated from the same study, so long as they reported on a different category of interest. For example, measurement or performance. However, if two publications reported on the same aspect of the study (for example, a published abstract followed by a full paper), only the most recent or complete publication was included. Registered trials and published abstracts were included only if there was no subsequent peer reviewed publication available. In cases where abstracts provided limited information, data was extracted from the corresponding Registered Trials where available. All titles, abstracts, and full texts were independently reviewed for inclusion by two reviewers (LMR and AN). Decisions were reached through discussion and by consensus, and any disagreements were reviewed with a senior researcher.

Data extraction and synthesis

Data was extracted by the first author (LMR). For publications where the first reviewer had uncertainties, a second reviewer (AN during initial review or KS during update) independently extracted data to validate the initial assessment. Missing or unclear data was recorded as “Not Reported” or “Unclear”. The screening process was facilitated by Covidence software, and the extracted data was compiled using MS Office Excel. Included publications were categorized according to the objectives of the review, and data was synthesized descriptively. It is important to note that the evaluation of measurement properties was based on publications with samples primarily consisting of Indigenous peoples.

The first author (LMR) worked with an Indigenous Elder (EJA) and an Indigenous research assistant (KS) who contributed to the interpretations and conclusions of the review. The Elder and Indigenous research assistant also advised on relevancy, strength-based language (writing in a good way), and future directions for research in this area. A further description of author position in relation to this work is available as an online resource (Supplemental Information).

Assessment of methodological quality

Given the objective of this review to describe the extent of study and use of PBMs with Indigenous people, it was not relevant to assess methodologic quality for all report types. The focus was on describing the frequency and purpose of PBM use rather than evaluating methodological rigor. However, ethically engaged research with Indigenous people was considered an indicator of quality. Ethical approaches to research involving or pertaining to Indigenous people is well established, whereby many institutions and regions have culturally and ethically appropriate research practices [25,26,27,28]. When assessing for ethical engagement with Indigenous people, we considered: (a) whether the publication reported some form of patient-oriented, community-oriented, Indigenous-centered, or otherwise engaged approach to research, or (b) whether the publication reported ethics approval from an Indigenous ethics committee. The presence of either of these elements was categorized as an “ethically engaged approach”. Of note, the Aboriginal and Torres Strait Islander quality assessment tool [29] was recently developed in Australia; as it has yet to be validated in global contexts, we chose not to use it in this review.

Results

PRISMA summary

This review includes 81 reports that met the inclusion criteria, originating from 63 unique studies. A PRISMA diagram is included in Fig. 1 PRISMA Flow Diagram. There has been a significant increase in reported use of PBMs with Indigenous people over the past five years, with 47 reports since 2018. Among these, the majority were peer-reviewed journal articles of original research (n = 45) [9, 11, 12, 14,15,16, 30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69] with an additional 9 registered trials [70,71,72,73,74,75,76,77,78], 1 pilot study [79], 11 abstracts [80,81,82,83,84,85,86,87,88], 1 dissertation [89], and 12 research protocols [10, 90,91,92,93,94,95,96,97].

Fig. 1
figure 1

PRISMA flow diagram. Figure generated using: Haddaway et al., [122]

Full size image

Geographical region

The majority of publications reporting the use of PBMs with Indigenous people were from Australia (n = 38) and New Zealand (n = 20). There were fewer reports from the USA (n = 9) and Canada (n = 6). There was a single multinational study, and several other studies from South Africa (n = 3), Ecuador (n = 1), and Mexico (n = 2) (Table 1).

Table 1 Characteristics of included reports
Full size table

Indigenous groups

At least 13 different Indigenous groups were identified. These included (as reported) Native American, American Indian, Native Hawaiian, Alaskan Native, Pacific Islander, First Nations (of Canada), Inuit, Metis people (of Canada), Māori (or New Zealand Māori), Indigenous Australians, Aboriginal and Torres Strait Islander, Indigenous Fijians, Tongans, Saraguara People of Ecuador, and Xhosa (Table 1). In some cases, the specific ethnicity beyond “Aboriginal” or “Indigenous” was not provided, however, the country is clearly stated. Among the included reports, only 26 described the methods used to determine ethnicity in the sample, such as self-report, or health records [10,11,12, 35, 42, 45, 48,49,50, 53, 55, 57, 61, 63, 69, 73, 75, 89, 95, 98,99,100].

Indirect and direct preference-based measures

Among the included reports, 37 identified the use of a PBM as their primary or one of their primary outcomes (Table 1); 38 reported it as a secondary outcome, and 6 were unclear or did not specify. The majority (n = 73) reported using indirect multi-attribute PBMs. Out of these, most (n = 69) reported the use of generic PBMs. Four publications reported the use of a direct PBM, with two utilizing standard gamble [65, 66], and the other two [33, 58] using the Visual Analogue Scale (VAS) to directly elicit health preferences. Overall, only 17 reported the use of a VAS, with 8 affirming the scale as 0–10 and the rest using a 0–100 scale.

The most commonly reported generic PBM was the EQ-5D (n = 50). Among these, 19 did not specify the versions used, while 17 used the EQ-5D-3L, 11 used the EQ-5D-5L, and 3 used the EQ-5D-Y. Other generic PBMs reported included the Child Health Utility 9-D (CHU-9D) (n = 4), Assessment of Quality of Life (AQoL) (n = 8), QWB (n = 3), Short Form Six-Dimension (SF-6D) v2 (n = 1), and the Health Utility Index Mark 3 (HUI3) (n = 4). Four condition-specific PBMs were reported in 6 publications—the National Eye Institute Visual Function Questionnaire-25 (NEI-VFQ-25) (n = 1), the Functional Assessment of Cancer Therapy-General Population (FACT-GP) (n = 1), the European Organization for the Research and Treatment of Cancer Quality of Life Questionnaire (EORTC-QLQ-C30) (n = 3), and a newly proposed oral-specific health utility scale (n = 1). Only 14 reports clearly stated which value set was used [11, 38, 43, 44, 54, 55, 73, 81, 82, 90, 93, 95, 101, 102], including value sets from Australia (n = 3), Canada (n = 1), New Zealand (n = 5), UK (n = 3), and USA (n = 2).

Purpose for using the PBM

Among the identified reports (Table 1), four examined the performance of a PBM [9, 41, 47, 52], four directly elicited health preferences [13, 58, 65, 66], four explored the development of a PBM [10, 64, 103, 104], two explored translation of a PBM [35, 68], while a majority (n = 71) used a PBM to measure health status. In some instances, reports could be classified in more than one category of use. For example, a research protocol from Australia aimed to develop a dental specific health utility scale (including preference elicitation) and measure dental HRQL within the same study. Among the included reports, 25 described interventional designs, 38 were observational, 5 were psychometric evaluations, 4 were economic evaluations, and 4 were longitudinal designs.

Nine of the included reports used a qualitative component or approach [30, 60, 64, 73, 78, 90, 102,103,104,105], with all but one [64] being part of a mixed methods study. Five reported using mixed methods, qualitative methods, or interviews but did not specify further the specific methodology used [60, 72, 78, 102, 103]. Of the remaining reports, one was a study protocol and described the plan to use phenomenological and collaborative storytelling to explore lived experience of injury-related disability [90]. A second report used phenomenology to explore the lived experience of asthma and assess a “whanau-centered, culturally tailored, health literacy-based intervention” among Māori in New Zealand [73]. A third publication described anthropological fieldwork with participant observation and interviews to contextualize their quality of life measure [30]. Howard et al. [104] published a protocol to develop a PBM for Indigenous Australians which planned to use Yarning Circles and semi-structured interviews. Three reports [64, 103, 104] described the use of Indigenous or decolonizing methodologies alongside other study methodology. For example, Willing et al. [64] reported a qualitative design situated within a Kaupapa Maori theoretical paradigm.

Performance of PBMs in Indigenous people

Four publications explored psychometric evaluation of a PBM specific to Indigenous people. Among these, 3 explored validity [9, 41, 52], including content validity [9, 52], discriminant validity [52], concurrent validity [41, 52], or convergent validity [52]. All four explored some form of reliability, including test–retest reliability [9, 41] and internal consistency [47, 52]. These studies were conducted in Australia [52], South Africa [41], New Zealand [9], and the USA [47].

Three of these studies used the EQ-5D [9, 41, 52]. The South African study found that the formally translated EQ-5D demonstrated both reliability and validity for Xhosa people [41]. Perkins (2004) suggested that the EQ-5D showed content validity but might lack construct validity among Māori people in New Zealand. The authors also suggested that the EQ-5D-3L had test–retest reliability with Māori people in New Zealand [9]. The most recent Australian study [52] concluded that the EQ-5D-5L demonstrated good concurrent validity, discriminant and convergent validity, and adequate internal consistency for Indigenous Australians. Only one publication explored a condition-specific based PBM, namely the NEI-VFQ-25 [47].

Direct preference elicitation with Indigenous people

We found two publications reporting the use of the standard gamble method with Indigenous people. One of these focused on population-based utility scores among Indigenous Australians for HPV infection and oropharyngeal squamous cell carcinoma [66], and the other reported population-based utility scores among Indigenous Australian women for HPV infection and cervical squamous cell carcinoma [65]. Both refer to one larger study around HPV. Two publications [9, 13] reported using data from a larger study on the valuation of health states with Māori people in New Zealand. In this study, valuations for health states were obtained using a visual analogue 0–100 scale (versus time trade-off or standard gamble). Similarly, a recent publication [58] used a VAS to value health states associated with suicide and depression. Although the VAS can be used as a direct measure of utility, it has a number of limitations [106]. Two publications reported obtaining individual direct preference weights using the Self-Evaluated Individual Quality of Life-Direct Weight (SEIQoL-DW) [30] and the Quality of My Life Questionnaire (QoML) [51].

Development of a PBM

We identified four reports that relate to the development of PBMs for Indigenous people. Arrow et al. (2018) published a protocol for a randomized controlled trial using a minimally invasive dentistry approach that proposed to simultaneously develop a dental specific child health utility scale for Aboriginal children in Australia [10]. Howard et al. (2020) also published a protocol to develop a new culturally relevant indirect, multi-attribute PBM including a descriptive system and underlying scoring algorithm for Aboriginal and Torres Strait Islanders in Australia [104]. Anderson et al. (2021) published an abstract reporting findings for the qualitative phase of this proposed study, finding a conceptual model of well-being including family, community, and culture that is being used to develop and score a well-being measure [103]. Willing et al. [64] describes a qualitative study situated within a Kaupapa Maori theoretical paradigm that specifically considers which key dimensions Western economic measures might miss, and to inform future development of a culturally-appropriate PBM. They report the need to “consider the individual within the context of the collective… and the environment” (p. 9).

Translation of a PBM

Translation was reported 9 times, of which three reported formal translations. Du Toit et al. [35] reported using the formal Xhosa and Afrikaans translation of the EORTC-QLQ-C30, a cancer-specific PBM. Jelsma et al. [41] reported using the Xhosa version of the EQ-5D (although not specified in the report, this would have been the three-level version, given that the five-level version was not yet available). The remaining studies suggested informal translation of the EQ-5D-3L to multiple Ghanaian dialects [53], the EQ-5D to Xhosa [39], and the AQoL to Creole [48]. Nagel et al. [105] reported forward translation of the EQ-5D-5L to multiple Australian Northern Territory languages to support a larger study [34, 91]. One report from New Zealand using the EQ-5D-3L indicated the use of “bilingual” translators to assist with a small proportion (11%) of interviews, but did not specify which language was used for translation (presumably Māori) [49].

Ethically engaged research involving Indigenous people

An ethically engaged approach to research was detected in 31 reports. Within 8 reports there was insufficient information to generate an opinion, although wording suggested it may have been considered. Most reports (n = 42) did not provide information related to engagement, relationship-building, or Indigenous-specific ethical approaches to research. The reports of ethically engaged research increased over time, with approximately 75% reported since 2018. Indigenous members of the team (EJA, KS) emphasized the value of understanding whether studies were done in a good way, and the importance of relational, engaged approaches to future research.

Discussion

Our review identified a substantial number of recent publications from diverse research areas that reported the use of PBMs with Indigenous people worldwide. This review also demonstrates that a wide variety of PBMs have been used to report health status, despite relatively little (or, in some cases, lack of) evidence on their performance in various Indigenous populations. Similarly, studies investigating translation or development of PBMs and preference elicitation were nearly absent, or in their early stages, often published as protocols. Studies involving Indigenous people were predominantly conducted in Australia and New Zealand, followed by the USA and Canada. There were only two reports exploring traditional direct preference elicitation methods (TTO or SG), while the majority of reports used indirect, multi-attribute PBM, the most common of which was the EQ-5D.

To our knowledge, this is the first systematic review to specifically explore the use of PBMs with Indigenous people, shedding light on their current application. Given the recent and relative increase in the number of studies reporting PBMs with Indigenous people, understanding performance metrics in this population is essential. Apart from a recent study of validity and reliability of the EQ-5D-5L for Indigenous Australians [52], studies evaluating performance are more than ten years old, presenting a possibly dated perspective of any of these instruments which have evolved to newer versions. Guidelines suggest first evaluating content and face validity of an instrument, followed by internal structure (construct validity), criterion validity, reliability and responsiveness [107]. Given this, it appears that for most colonized countries, a comprehensive evaluation of the performance of any PBM with Indigenous people should be done before using PBMs with these populations. COSMIN’s guidance further affirms the need to start with exploration of content validity and other forms of validity for any PBM that may be used to measure HRQL with Indigenous people [108].

There is a significant and growing body of Indigenous research and work published in academic journals that shares Indigenous perspectives on health, wellness, quality of life, and health related quality of life [2, 109,110,111]. This includes development of Aboriginal or Indigenous specific HRQL measures, such as the Aboriginal Children’s Health and Wellbeing Measure [112, 113]. It also includes accounts of what current PBMs of health-related quality of life (based on western biomedical models of health) fail to capture, such as elements of community, the environment, or spirituality [2, 29, 64, 110, 111, 114]. This review provides preliminary evidence of dimensions of health that may not be adequately captured for Maori people [64], as well as for Aboriginal and Torres Strait Islander people [103], emphasizing the need to integrate this knowledge into the exploration of validity and development of PBMs that are appropriate, accurate, valid, and reliable for Indigenous Peoples.

We did not find many reports related to the development or translation of PBMs to Indigenous contexts—Jelsma et al. [41] studied the performance of a Xhosa translation of the EQ-5D in South Africa, but the scarcity of translation exercises is noted. In communities in which traditional Indigenous language is still the primary language, translation of PBMs may be an important strategy to include traditional language speakers. Both Arrow et al. [10] and Howard et al. [104] proposed studies to develop a dental specific health utility scale and a preference-based wellbeing measure (respectively) in Australia. There are also recent efforts being made to generate an Indigenous specific PBM using Indigenous or decolonizing methodologies [64, 103, 104]. This approach should be modeled in the pursuit of similar questions around developing PBMs in other colonized countries. However, if Indigenous-specific HRQL instruments exist, perhaps there is an opportunity to investigate the appropriateness of deriving preferences for existing Indigenous-specific instruments.

The review by Angell et al. [8] focused on measures of HRQL with Indigenous people, but only captured one PBM. There are multiple factors that may have contributed to this recent increase. Firstly, there is a growing demand and interest in the application of PBMs in general, as they have become a common metric in evaluation of health and health system performance. Additionally, the global focus on reconciliation efforts prompted a shift in research practices to recognize, include, and partner with Indigenous people, aiming to address historic harms. The increasing number of studies in this review involving or reporting Indigenous people may be a result of this political and organizational shift towards engagement and reconciliation. In any case, we believe the inclusion of Indigenous people in studies using and reporting on PBMs is a beginning to the necessary representation and power shift in how research on PBMs is conducted or reported in Indigenous communities.

It is important to note that the majority of reports in this review did not clearly indicate whether the work was undertaken using currently accepted approaches for ethically engaged Indigenous research. Engaged approaches can reflect researchers' intentionality and their acknowledgment (or lack thereof) of the historical harms that colonizing research practices have inflicted upon Indigenous people. The lack of reporting likely calls for ongoing efforts to increase knowledge, awareness, and training in ethically engaged Indigenous research. It is also possible that academic journals publishing on PBMs are based in positivist epistemologies and are still maturing in their willingness and ability to include detailed information on engaged scholarship. Consequently, the records we found may not have provided comprehensive details regarding the methodology, particularly in terms of the extent of engagement with Indigenous communities.

PBMs, such as the EQ-5D, have already been adopted by some health systems to support health care resource allocation and economic evaluation [115]. The limited amount of recent information on performance of PBMs with Indigenous people suggests that the use of PBMs with Indigenous people should be applied with caution, and warrants consultation with Indigenous Peoples. There is recent, ethically engaged evidence of both validity (content, construct, criterion) and internal consistency of the EQ-5D-5L in Aboriginal and Torres Strait Islanders [52]. However, although this may indicate support for the EQ-5D-5L (at least in Aboriginal and Torres Strait Islanders), the authors also explicitly state that content validity was endorsed “in the absence of another suitable instrument being available” [52]. This aligns with concerns nearly two decades earlier about construct validity in Māori [9] that may suggest further exploration of fundamental questions of validity, and what constitutes health for Indigenous people. Decision-makers and policymakers may wish to exercise caution in their choice of PBM, and if they choose the EQ-5D, consider the transferability of Ribeiro’s findings to specific Indigenous communities in their region.

We undertook a very broad search strategy, in terms of populations, measures, study objectives and designs, intending to understand the full scope of use of PBMs with Indigenous populations. However, we were only able to include studies published in English. This review may also have failed to capture studies reporting broader measures of well-being, or studies supporting the early development of measures, such as theoretical or conceptual studies that may have preceded PBM-specific terminology. Due to the diversity of types of studies in this review, however, we chose not to evaluate the quality and strength of the included studies.

This work focused on describing the state of the peer-reviewed literature for PBMs with Indigenous people and explored how Indigenous groups and communities engaged in the work. This resource can be used to inform future work, and how research teams might engage with Indigenous partners in a research agenda on this topic. This review is based on the intention to understand how current PBMs are used and how they perform, such that they can be used, or put aside, appropriately.

This review suggests that further work is required to evaluate the performance of PBMs with Indigenous people. Given the predominant use of indirect PBMs, it is important to assess not only the validity of health status descriptive systems, but also the concept of valuation and preference elicitation. Given the limited evidence on the performance of current PBMs, future research might also focus on development of a PBM from an Indigenous-specific HRQL descriptive system and preference valuation that is culturally appropriate for Indigenous people. Work in this direction should only be pursued in an engaged manner, therefore relationship building between Indigenous communities, health economists and health policy actors would be beneficial. Lastly, exploration of the relevance of the Aboriginal and Torres Strait Islander quality assessment tool to other Indigenous contexts should be explored to support future systematic reviews and assessment of research.

Future research is not without ethical and political obligations for western researchers to recognize spaces for Indigenous self-determination in research, actively engaging in the work of decolonizing current (colonizing) research and policy systems [116, 117]. Similarly, theoretical assumptions of the health economic paradigm itself should be considered in terms of their euro-western roots and documented limitations [118], and the relation to Indigenous ways of knowing and Indigenous approaches to decision-making and priority-setting [119,120,121].

Conclusion

PBMs have many applications, including assessment of population health and informing economic evaluation, and therefore resource allocation decisions, in many countries, including Canada. This review provides insight on the current breadth of use of PBMs worldwide with Indigenous people, as well as information on their validity and reliability. Understanding the validity and performance of PBMs with Indigenous populations is particularly important to better understand how such measures, or other measures, might be used in economic analysis that inform resource allocation decisions that affect Indigenous populations. Perhaps most importantly, it is our hope that this review can also facilitate meaningful conversations and work towards accurate and appropriate measurement of HRQL, particularly given the need to address health inequalities experienced by Indigenous people.