Adaptation and Psychometric Properties of the Spanish Version of the Supports Intensity Scale (SIS)

Considerable research has been conducted internationally on the psychometric properties and use of the Supports Intensity Scale—SIS (Thompson et al., 2004). The emergence internationally of an ecological model of disability and the related supports paradigm (Luckasson et al., 1992, 2002) has resulted in the translation of the SIS into 13 languages. This extensive interest and use have raised questions regarding its etic (universal) and emic (culture-bound) properties as well as the need to demonstrate its utility and psychometric properties across cultural and linguistic groups. Confirmation of the reliability and validity of the SIS has been published for the English version (Thompson et al., 2004; Thompson, Tassé, & McLaughlin, 2008; Wehmeyer et al., 2009), the French version (Lamoureux-Hebert & Morin, 2009), and the Dutch version (Buntinx, 2008; Claes, van Hoove, van Loon, Vandervelde, & Schalock, 2009). Our purpose in the present study was to evaluate the reliability and validity of a Spanish version of the SIS on a Spanish sample.

We administered the Spanish version of the SIS to 885 people (538 males [61.5%] and 337 females [38.5%]). They ranged in age from 15 to 76 years (M  =  34.60, SD  =  11.77). All participants had a primary diagnosis of intellectual disability (with 91% having an assessed IQ of 51 or below), with 15% also having speech and language impairment. In regard to the assessed level of adaptive behavior, 43% of the sample showed a mild level; 40%, a moderate level; and the other 16%, a severe or profound level according to the scores obtained on the Inventory for Client and Agency Planning—ICAP (Bruininks et al., 1986; Montero, 1996, 1999). Ten percent of the sample also had one or more of the following disabilities: mobility limitations, psychiatric impairment, visual impairment, and neurological damage. More than two thirds of the participants (68%) lived in an urban zone, with 32% living in a rural zone.

The version of the SIS evaluated in this study was the Spanish version developed by Verdugo, Arias, and Ibáñez (2007). The SIS represents a multidimensional measure designed to determine the profile and the intensity of the support needs of an adult with intellectual disability (Thompson et al., 2004). The scale is divided into three sections. Section 1, Needs Support Scale, consists of 49 activities divided into six subscales: Home Living, Community Living, Lifelong Learning, Employment, Health and Safety, and Social Activities. Section 2, Protection and Advocacy Supplementary Scale, is not used to determine the Support Needs Index, but includes critical support needs related to health and safety, self-advocacy, and personal protection. In both sections each activity is ranked according to frequency, daily support time, and type of support. Section 3, Exceptional Medical and Behavioral Support Needs, lists 15 medical conditions and 13 behavioral problems that are assessed on a 3-point Likert scale regarding the need to either monitor or maintain the medical condition (Section 3A) or prevent the occurrence of the listed behavioral challenge (Section 3B).

To provide data for criterion-related validity, we developed an informal evaluation tool to measure support needs (as was done by the original authors of the SIS). The present scale had a Likert format, and the required task was to classify the support needs of each participant in reference to each area measured by the SIS. Ratings ranged from 1 (lowest support needs) to 5 (highest support needs).

Scores on the ICAP (Montero, 1996, 1999) were used as a basis for determining criterion-related validity. The ICAP includes two normed measurement instruments, one for adaptive behavior and the other for maladaptive behavior. It also provides a Service Score index, which uses the measurements of adaptive and maladaptive behavior to indicate the overall level of care, supervision, or training a person requires. Several psychometric studies have shown that the ICAP has adequate reliability and good content and construct validity (Montero, 1996).

The first stage in the cross-cultural adaptation process was the forward translation. Two English–Spanish translations were made by two professional psychologists with a good command of English and whose first language was Spanish. We used guidelines proposed by the International Tests Commission (Hambleton, 1996; Tanzer & Sim, 1999) and other guidelines to the cross-cultural adaptation of tests (Beaton, Bombardier, Guillemin, & Ferraz, 2000). Any disagreements between the two translators were reconciled and synthesized through a face-to-face work session.

A native English translator converted the initial translated version back into the original language. The scope of this process was to ensure that the translated version reflected the same item content as the original version. Results indicated that it did.

A group of nine experts with considerable experience internationally in assessment consolidated a field test version of the Spanish SIS that was based on the initial translation (Step 1). The adequacy of the Spanish translation was determined by an agreement analysis among the assessments carried out by the nine expert judges.

This field test of the Spanish version was applied to 30 people with intellectual disability, which ensured that the adapted version was still retaining its equivalence in an applied situation.

The final version of the Spanish SIS was developed by four academic researchers from the expert committee mentioned in Step 3 (Verdugo, Arias, Ibáñez, & Gómez, 2006; Verdugo, Ibáñez, & Arias, 2007). This new group became the coordinating committee for appraisal of the adaptation process and has been in charge of this assessment tool since 2007.

The adaptation of an assessment tool can affect both its reliability and validity. Once the adaptation process was finished as described above, we recruited individuals from the voluntary collaboration of several developmental disability centers located in 20 provinces of Spain. The SIS interviewers at each center were qualified professionals who had already been trained to reliably administer the SIS. As part of the administration process, we also asked the interviewers to complete the subjective evaluation form of the level of the needed supports and collect the most recent ICAP evaluation results for each participant. This information was useful for analyzing construct and criterion-related validity. In some cases, we also asked for two applications of the SIS 3 weeks apart and two completed SIS protocols obtained on the same person with intellectual disability done by two different professionals.

The statistics packages we used to analyze the data were SAS, version 9.1.3 (The SAS Institute, 2005); SPSS, version 14.0 (SPSS, 2006), and STATISTICA, version 7.0 (StatSoft, 2005).

We calculated Cronbach's alpha coefficients in the different ranges of age (16 to 19, 20 to 29, 30 to 39, 40 to 49, 50 to 59, 60 or more). In all the cases, the internal consistency coefficients were very high, exceeding the absolute value of .90. The coefficients ranged from .903 to .995, which indicate that the internal consistency is excellent according to guidelines provided by Landis and Koch (1977): .00 is no agreement; .01 to .20, insignificant; .21 to .40, a fair level of agreement; .41 to .60, moderate, .61 to .80, good; .81 to 1.00, very good.

Split-half reliability: After randomly dividing the items into another set of split-halves and recomputing them, we calculated different indexes of reliability: the Cronbach's alpha coefficient for each half, the Pearson product-moment correlation coefficient between the first and the second half, the corrected correlation coefficient for reducing the attenuation, the global coefficients reliability (split-half and Guttman), and the average correlation among the items. As shown in Table 1, in all cases the global coefficients of reliability were above the threshold of .90.

The same interviewer applied the SIS to a sample of 143 participants at two different times with an interval of 3 weeks. We calculated the Pearson's product-moment correlation coefficients between the test and retest scores and obtained values between .84 and .93 (between .901 and .981 in the corrected correlation). These coefficients are excellent according to the guidelines provided by Cicchetti and Sparrow (1981). Specifically, the test–retest coefficients for each of the 49 items ranged from .63 (Item 4 of Health and Safety Activities, ambulating and moving about) to .90 (Item 3 of Home Living Activities, preparing food). Using the guidelines provided by Cicchetti and Sparrow for evaluating the reliability coefficients, we found that 46 coefficients (93.9%) fell within the excellent range, and only 3 (6.12%) in the good range, which guaranteed the test–retest reliability of the SIS.

The interrater reliability was evaluated by two professionals who applied each scale twice, with an interval of 2 weeks. Pearson product-moment correlation (and corrected) coefficients were computed. Results of the different subscales ranged from .60 and .84 (.62 and .86 in the corrected correlations). These data are consistent with those reported by Thompson et al. (2004).

The adequacy of the Spanish translation was determined by an agreement analysis among the assessments carried out by expert judges. Although we calculated the level of agreement among judges with different types of indexes (see Tables 2 and 3), the observer agreement chart (Bangdiwala, 1987; Friendly, 2001) provides a simple graphic representation of the strength of agreement in a contingency table and a measure of strength of agreement with an intuitive interpretation (see Figure 1).

The Bangdiwala's agreement chart shows the information about two aspects of the data: (a) the level of agreement (the global magnitude of agreement in the contingency table); and (b) the level of partial agreement (distance among observers—or judges—in the categorization of events). The agreement chart is constructed as an n × n square, where n is the total sample size (i.e., the nine domains or support areas). Black squares, each of size n_ii × n_ii, show the observed agreement. These are positioned within larger rectangles, each of size n_i+ × n_+i. The large rectangle shows the maximum possible agreement given the marginal totals. Thus, a visual impression of the strength of agreement (Friendly, 2001) is determined by:

To determine the degree of judgments reliability, we used the guidelines provided by Landis and Koch (1977). As shown in Tables 2 through 4, the degree of agreement between judges fell in all the cases (with the exception of the Bangdiwala's raw coefficient) in the good category. The Bangdiwala's agreement coefficient for each domain of the SIS is presented with the Bangdiwala's agreement statistics shown in Figure 1 (Bangdiwala, 1987; Friendly, 2001). These results support the content validity of the Spanish version of the SIS.

After dividing the participants into three groups (delimited by Quartiles 1 and 3 in the Support Needs Index of the SIS), we verified the differences between the average ranks of each item according to which group (high, medium, low) each participant had been classified in. In view of the ordinal nature of the data, we used a Kruskal-Wallis one-way analysis of variance (ANOVA) by ranks. The results of the contrast (chi-square) were highly significant, p < .0001, for all items, providing evidence of the discriminating power of items.

We calculated the corrected homogeneity index and the averages of the abovementioned coefficient by subscale considering each one of the segments of age. The results exceeded the value of .75 in all cases. The rank of the coefficients ranged from .78 to .97 (the median one was situated between .83 and .93). As a consequence, the subscales demonstrated a high consistency through the different segments of age (according to the ranges of clinical significance recommended by Cicchetti, 1994).

We use data from the subjective evaluation of the level of needed supports to establish criterion-related validity. We computed the polychoric correlations matrix of items. All of these coefficients ranged from .64 to .93; therefore, we concluded that they were significant because they exceeded the threshold of .35, usually considered as representative of the criterion-related validity.

The variance analysis and the post hoc comparisons by pairs in the different subscales according to the six age groupings revealed no significant differences among the different age-related scores, p > .01.

Because both the SIS subscales and the composite scorings measure aspects related to the support needs, they are supposed to correlate significantly among them. After calculating the correlation matrix of the normed scores of the subscales and the composite scores, we found that the obtained coefficients among subscales ranged from .78 to .88, with subscale total scores ranging from .90 to .95.

We obtained a significant correlation between SIS subscales scores and the levels of intellectual functioning. The results show ρ coefficients between −.417 and −.524, p  =  .000, which reveals an inverse relationship. We completed this analysis with a unifactorial variance analysis, with the factor being the intellectual disability of the participants and the dependent variables, the typified subscales scores. All the resulting contrasts were significant, p < .01. The post-hoc intergroup comparison test showed the existence of significant differences regarding the disability level, p < .01, whereas the intragroup differences among the diverse subscales were minimal. These results support the instrument's construct validity.

The correlation coefficients obtained between SIS and ICAP scores ranged from −.498 and −.589, p  =  .000, n  =  152. These significant negative correlation coefficients are consistent with the previously referenced psychometric studies. In addition, we developed a unifactorial variance analysis. The dependent variables were the typified subscales scores and the SIS total and the factor was group membership (grouped on mild, moderate, or severe adaptive behavior). This unifactorial ANOVA revealed the existence of significant differences among the three groups in all SIS subscales and the lack of influence of the intragroup differences.

Our predictor variable was the raw scores of the SIS interviewees and grouping variable was the categories (mild, moderate or severe), according to the individual's adaptive behavior score. We conducted the analysis on a sample of 147 participants by group. The intragroup correlations of each variable with the canonic function were, in a decreasing order, .901 (Home Living Activities), .871 (Lifelong Learning Activities), .801 (Employment Activities), .800 (Health and Safety Activities), .793 (Social Activities), and .788 (Community Living Activities). In total, the function correctly classified 61.9% of cases, and it was more effective in classifying the participants with a severe level of adaptive behavior (72%) than those with a mild (64.5%) or moderate deficit (55%).

Reliability analyses reported in this article demonstrate the excellent internal consistency and test–retest reliability of the SIS, consistent with the results on the English, French, Italian, Catalan, and Dutch versions (Claes et al., 2009; Lamoureux-Hébert & Morin, 2009; Schalock, Thompson, & Tassé, 2008a, 2008b; Thompson et al., 2004, 2008).

Similarly, the three types of validity reported supported the validity of the instrument. Content validity was based on the agreement of expert judges about the adequate correspondence between items and their respective subscales. The degree of agreement was good in all the cases, and the discriminative power of items was confirmed. Finally, the indexes of the subscales corrected homogeneity were high in all cases. Criterion validity was based on the correlation between SIS subscales and rater estimates of support needs, according to the procedure followed by the original authors of the SIS. The results were positive and representative of this type of validity. Construct validity indicated that (a) there was no age differentiation in the SIS scores, (b) there was a significant intercorrelation among SIS scores, (c) there were significant correlations between SIS subscales scores and both the intellectual functioning level and adaptive behavior assessed, and (d) there were significant differences in SIS scores among those individuals rated on the ICAP as mild, moderate or severe. These results are consistent with those published articles regarding the SIS referenced above.

In the future, we anticipate that SIS-related information will be used at three levels within Spain: the individual, the agency, and the wider system. At the individual level, the Spanish version of the SIS will contribute to the development of individualized plans and strategies centered on each person's assessed support needs rather than on planning from the global perspective of services being received by the person. Both the supports paradigm and the quality of life provide the necessary elements to develop person-centered planning strategies that address the reduction of support needs and the improvement of quality of life (Schalock & Verdugo, 2002; Schalock, Keith, Verdugo, & Gómez, 2009). At the wider system level, it would be good to rethink how to guide resource allocation and how to implement the new paradigm of supports.

Finally, the process of developing a measurement tool is always an unfinished task. In that regard, we would like to contribute to enhancing the use of this tool in Spain. Two suggestions apply here: First, we think that the good results regarding the reliability of the Supplemental Protection and Advocacy Scale (Section 2) found in this study suggest the need to rethink the place of this scale in the SIS. We consider that the information given in Section 2 is essential for promoting full and lifelong inclusion. Thus, this section should be returned to the Support Needs Scale (Section 1) and considered in the computation of the SIS Support Needs Index. Second, according to our experience in this study and findings in studies about interviewing people with intellectual disabilities (e.g., Finlay & Lyons, 2001), there are a number of difficulties that arise when questions concern quantitative judgments, abstract concepts, inferences, and generalizations (Claes et al., 2009). Therefore, if SIS involves the consumer as a vital source of information, we suggest that the contents and form should be adapted to the different levels of understanding of people with intellectual disability.

Finally, we recognize that this study has some limitations. The first is the sampling method. We used a nonprobabilistic sample, based on the volunteer centers, instead of a random sample of centers. Although the sample was very large (875 valid participants), it does not guarantee the generalization of the data. A second limitation is that interviewers only interviewed one respondent when completing the SIS instead of the suggestion made in the SIS Users Manual that “at least two respondents be interviewed to obtain accurate and complete information” (Thompson et al., 2004, p. 23). Consequently, the multiple respondents could change our reliability findings. This should be addressed in future research.

We thank the FEAPS professionals who helped in the application of this assessment tool.