Phase 1: Item generation and development
Items were generated from two sources for content validity: i) the research team and ii) healthcare professionals. The research team (SWH, AD, JH) generated items from reviewing existing mobility scales, discussion with experts in mobility among people with dementia and a search of healthcare textbooks on clinical skills for mobility aid training. The project was conducted in accordance to the Declaration of Helsinki and was approved by the <blinded> Research Ethics Board for Health Sciences Research Involving Human Subjects.
Two focus groups of healthcare professionals from relevant disciplines in <blinded> providing care across health settings for people with dementia (n = 12; 1 geriatrician, 2 registered nurses, 5 physical therapists and 4 occupational therapists) were also completed. The healthcare professionals were working in the areas of acute hospital care, community care, long-term care, day hospital and rehabilitation hospital. Eleven (92%) participants had 10 or more years of total clinical experience working in geriatrics. All healthcare professionals provided written informed consent prior to the start of data collection.
Our focus groups identified use of a 4-wheeled walker as the most common mobility aid used in this patient population. Therefore, items were sought that 1) represented the minimum functional tasks needed to assess and evaluate independence and safety with the use of a 4-wheeled walker, and 2) the components within each task that can be objectively evaluated for function and to represent safe use.
The focus groups and work by the research team generated 11 tasks and components within each task. To render the scale manageable in the clinical and research setting, the authors sought to reduce the number of items. The information was presented to an independent panel of five healthcare professionals (1 registered nurse, 2 physical therapist, and 2 occupational therapists) with more than 10 years of clinical experience in geriatrics and who had not participated in the first set of focus groups. The participants were asked, “Rank how important it is to include each task, regardless of setting, when assessing physical function and safety with a 4-wheeled walker”. The 11 identified tasks were ranked on a 5-point Likert scale with a score of 1 representing “least important” and 5 representing “most important”. The scores for each task were summed and averaged across raters. Tasks were kept if the average score was rated ≥ 4, indicative of the item being rated as either important or most important. (Table 1)
The evaluation resulted in a final selection of nine tasks with task components that comprise The Safe Use of Mobility Aid Checklist (SUMAC). The research team categorized the components in each task into two separate areas of focus, physical function (PF) which includes items related to an individual’s physical ability to perform the task (e.g., stand independently) and their use of the equipment (EQ) which includes items related to an individual’s safety in using their 4-wheeled walker (e.g., brakes engaged on walker). Items for PF and EQ are not equally present across the tasks and reflect to some extent the difficulty of the activity. The distribution of scoring items for each task (number of PF items/number of EQ items) are: sit to stand (3/7); pivot turn to sit in chair (3/7); walking on level surface (11/6); walking with horizontal head turns (1/6); walking with concurrent cognitive task (1/6); walking around obstacles—figure of eight (1/6); walk through an open doorway (1/5); open, walk through and close door that opens away from the person (1/10); and open, walk through and close door that opens in to person (1/10). Physical function items are rated on a 3-point scale (0, 1, 2) and use of equipment is a dichotomous scale, where the item is scored as observed yes (1) or no (0). Total PF scores can range from 0 to 38 and the total EQ score can range from 0 to 63. A higher score in each category indicates better physical function and safety.
Phase 2: Reliability and validity evaluation
This phase of the study involved the recruitment of people living with dementia and a separate sample of health care professionals. The project was conducted in accordance to the Declaration of Helsinki and was approved by the <blinded> Research Ethics Board for Health Sciences Research Involving Human Subjects.
People with dementia who used a 4-wheeled walker for ambulation were recruited from a local day program to be evaluated with the SUMAC. Participants had a diagnosis of probable AD from a geriatrician based on the National Institute of Neurologic and Communicative Disorders and Stroke-AD and Related Disorders Association (NINCDS-ARDRA) criteria.[18] Inclusion criteria were: 50 years of age and older, English proficiency, able to follow instructions, mild to moderate disease severity, able to walk 60 meters without support from another person and have a substitute decision maker, who were all family members of the person with dementia, to provide information about health and daily activities. Exclusion criteria were any muscle and/or nerve problem that limited movement. Informed written consent was provided by either the participant or their substitute decision maker, who provided informed consent on their behalf and then the participant provided assent to participate in the study.
Demographic and clinical information collected on the participants included age, gender, falls in the previous 12 months, number of prescription medications, number of comorbidities and instrumental activities of daily living, as per the Lawton-Brody Instrumental Activities of Daily Living and Basic Activities of Daily Living scales [19]. Participants also completed the Iconographical-Falls Efficacy Scale, this scale has been validated in older adults with cognitive impairment [20]. Participants were videotaped while performing the nine tasks of the SUMAC.
The raters consisted of a convenience sample of five healthcare professionals (5 physiotherapists). All healthcare professionals provided written informed consent prior to the start of data collection. The healthcare professionals were working in the areas of acute hospital care, community care, and long-term care. Two people (40%) had had 10 or more years of total clinical experience working in geriatrics. The inclusion criteria were: registered healthcare professionals with experience working with older adults with dementia. Each assessor attended a one hour long one-on-one training session on the use of the SUMAC. In the training session, each person received a copy of the assessment tool and was instructed in the use of the tool. The last activity in the training session was for each healthcare professional to evaluate videos of a person performing the nine tasks of the SUMAC.
Reliability
In the reliability evaluation, each of the healthcare professionals was asked to view the 10 participant videos and evaluate them using the SUMAC on two occasions. The two visits were set one week apart. In both visits the viewing order of each participant was randomized, but all components of the SUMAC were presented together and in the order in the tool for each participant.
Validity
Our sample of five HCP who participated in the reliability study of the SUMAC also completed the gait subscale of the Performance-Oriented Mobility Assessment (POMA) on each participant. Criterion validity of the PF and EQ scores of the SUMAC was evaluated against the Performance-Oriented Mobility Assessment (POMA) scores.[21] The POMA is a reliable tool in people with dementia [22]
For the evaluation of construct validity, a panel of eight healthcare professionals with clinical experience working with older adults and people with dementia (6 physical therapists, 2 registered kinesiologists) and who had not participated in any aspect of the development or reliability evaluations of the SUMAC was assembled. All members of the panel had the same training session as the healthcare professionals in the reliability study. The panel viewed the participant videos in a group setting in a single session with discussion until consensus was reached in scoring the physical function and safe use of the equipment.
Data Analysis
a) Reliability: Values for relative and absolute reliability were calculated. An a priori sample size calculation (α = 0.05 and β = 0.20) for the reliability study indicated that 10 participants and 5 assessors making 2 evaluations would be needed if a target ICC of 0.90 was desired.[23] This sample size minimized recruitment and participant burden while optimizing the use of multiple healthcare professionals as assessors of the videos.
The relative reliability values of inter-rater and test-retest reliability were calculated for the PF and EQ domain scores of the SUMAC using the intra-class correlation coefficient (ICC). Repeated measurements by different raters on the same day were used to calculate inter-rater reliability, while repeated measurements by the same rater on different days were used to calculate test-retest reliability. An ICC value of greater than 0.90 is considered excellent, between 0.80 to 0.89 are good, 0.70 to 0.79 are fair, and values less than 0.70 are of questionable clinical value [24].
Two measures of absolute reliability were calculated: standard error of measurement (SEM) and minimal detectible change with a 95% confidence interval (MDC95) for the PF and EQ domain scores of the SUMAC. The SEM is the measurement error associated with a single value and is expressed in the same units as the scale [24]. The smaller the SEM, the greater the absolute reliability. The MDC95 is an estimate of the smallest change in the score that can be detected beyond measurement error [25]. It is also measured in the same units as the measurement scale. For the present study, the SEM was calculated using pooled standard deviation (SD) and ICC values for each group. Calculations of SEM and MDC95 were:
b) Validity: Spearman’s rank-order correlations were used to assess construct validity between each healthcare professional and the panel consensus scores. Each participant’s mean for each subscore and a total using the minimum and maximum scores from among the HCP’s assessment were calculated for both SUMAC-PF and SUMAC-EQ and compared to the consensus score of the panel.
Spearman’ rank-order correlation analysis was also used to evaluate criterion validity between the gait component of the POMA, and the SUMAC-PF and SUMAC-EQ scores. We hypothesized that the POMA score would be moderately correlated with the SUMAC-PF score and not correlated with the SUMCA-EQ score. All statistical analysis was performed using SPSS version 25.0 (IBM Inc., Chicago, IL, USA). Statistical significance was set at p<0.05.