Normative data for Lezak’s Tinkertoy test in healthy Italian adults

Introduction

Executive functions are the cognitive capacities that control lower-level functions and are essential to future-oriented thought and behaviour. They are affected by head injuries²⁴ or arise because of a focal frontal lesion, either cortical^5,24,25 or subcortical¹³. In particular, the term executive functions refers to cognitive, emotional and behavioural aspects of conduct involved in achieving a specific purpose. Executive functions include processes that are complex, mixed together and in constant interaction. They facilitate the optimum adaptation of the individual to the environment^1,16,23,24. Lezak¹⁶ suggested the division of competencies into four specific components: volition, planning, goal-oriented behaviour and effective performance. Several psychometric instruments are available for evaluating executive functions during neuropsychological examinations. However, most of them are generally highly structured (the task and the stimuli set the goal and the processes required to achieve that goal)^15,16. Moreover, none of the tools currently in use for evaluating the performance in the domain of executive functions is able to assess how the patients are able to formulate a goal and to plan how to pursue it, which are prerequisites for a return to work as well as for social life.

The Tinkertoy was created originally as a toy for kids made of various wooden and plastic pieces (wooden dowels, knobs, wheels, connectors, caps, points), to be assembled freely in order to make constructions. Based on this toy, Lezak created a test for the neuropsychological assessment of executive functions, which gives subjects the opportunity to use their own initiative and does not force them to choose from a series of predetermined alternatives. In fact, one of the most relevant characteristics of frontal lobe syndrome is an environment-dependent behaviour, which makes it difficult to cope with the requirements of everyday life. In this respect, Lezak’s Tinkertoy test (TTT) stands out, because it was specifically conceived to examine the ability to generate the most achievable goal, to organize, to plan and act, and to respond in a flexible way in a given context¹⁵. At the outset, studies of the TTT showed that it could be considered a useful predictor of employability^3,15. Particularly, some researchers found the TTT complexity score correlated more positively with the employability of traumatic brain-damaged patients, than other tests for executive functions, such as trail making test -B, maze tracking, and several WAIS-R subtests^9,10. Ownsworth and Shum²⁰ showed that the difference in TTT scores between employed and unemployed patients after strokes was highly significant (p < 0.005 in a group of 27 subjects.) According to the authors, the TTT seems to describe productivity outcomes better than other tests of executive functioning (i.e. the FAS test and the five -point test), independent of the presence of hemiplegia and the elapsed time since the stroke. Furthermore, the TTT has been shown to be useful both for differentiating between types of dementia and for evaluating the severity of dementia^12,17. In addition, subjects generally find the TTT interesting or amusing, so that it is easy to carry out this test, even in the cases of patients who are not very cooperative. Despite the fact that the TTT is commonly used in a clinical context, the only normative data refers to a very small sample of non-Italians¹⁶. Given both the potential relevance of this instrument for neuropsychological practice, and the lack of any validation so far, the present study aimed at setting TTT normative values in Italian adults, in order to determine firstly inferential cut-off points and their tolerance limits, and then equivalent scores, applying a statistical technique developed for neuropsychological tests^7,8,22.

Materials and methods

Stimuli and procedure

Test items, administration procedure and scoring criteria in this study followed the ones described by Lezak in the fourth edition of the Neuropsychological Assessment Handbook¹⁶. The test items were selected from the classic version of the Tinkertoy set. Namely, 50 items were used: 24 wooden dowels (4 red, 4 green, 4 orange, 6 blue, 6 yellow), 10 wooden knobs, 4 wooden wheels, 4 wooden caps, 4 wooden connectors, and 4 plastic yellow connectors (Figure 1).

Figure 1. Tinkertoy items used by Lezak¹⁶ for the Tinkertoy test.

Each subject was individually presented with the aforementioned 50 pieces in different colours and forms, placed at random on a clean surface, and were told to build up whatever construction they wanted with a 5-minute minimum time limit, but no maximum time limit. On completion, the subjects were asked to say what the construction represented. Assessment took into account 7 performance variables: 1. Made construction(s) - whether the subject made any combination of pieces; 2. Number of pieces - total number of pieces used; 3. Name - whether and when the subject gave a name appropriate to the construction’s appearance; 4. Mobility (wheels working) and moving parts; 5. Three-dimensionality - whether the subject’s construction had three dimensions; 6. Free-standing - whether the subject’s construction stayed standing; 7. Errors - pieces forced together (misfit), connections not properly made (incomplete fit), and dropped and not picked up pieces (see Table 2). At the end a complexity score was given, determined by the sum of the points earned in each of the performance variables, with a maximum of 12 points (for two examples, see Figure 2).

Table 2. TTT scoring criteria.

Variables	Scoring Criteria	Points
1.  Made construction(s) MD	whether S makes any combination of pieces	1
2.  Number of pieces NP	total number of pieces used	•  1 = NP ≤20 •  2 = NP ≤30 •  3 = NP ≤40 •  4 = NP ≤50
3.  Name N	whether and when S gives a name appropriate to the construction’s appearance	•  0 = none N •  1 = description or    post hoc naming •  2 = vague or   inappropriate N •  3 = appropriate N
4.  Mobility M a) and      moving parts M b)	•  M a): working wheels •  M b): moving parts	•  M a) = 1 •  M b) = 1 (maximum: 2 points)
5.  Tridimensionality 3D	whether the construction has three dimensions	1
6.  Free-standing S	whether the construction stays standing	1
7.  Errors E	•  misfit: pieces forced together •  incomplete fit: connections not properly made •  dropped and not picked up pieces	-1 for each error

Figure 2. Two examples of performance at the TTT by a neurologically unimpaired subject and a TBI patient recruited exclusively for this comparison.

The first is a male, 46 years old, with 13 years of education; his performance has been evaluated as 11.64 (corrected score) and 4 as equivalent score, according to Table 2. The TBI patient is a female, 36 years old, with 13 years of education; her performance has been evaluated as 4.33 (corrected score) and 0 as equivalent score.

Results

Normality criteria are generally appraised by comparing one subject’s performance to that of all the other subjects. This implies homogeneity across the subjects in the comparison, and hence imposes the requirement that all possible factors influencing performance have been taken into account and removed from the raw scores. From a statistical point of view, this aim can be accomplished using stratification, which nonetheless, in some cases, raises problems concerning the sample size in each stratum. Alternatively, the effect of confounding factors can be removed from raw scores in a multiple regression model^4,8,22. In order to set correction grids for the raw values of participants’ complexity scores, a linear model for the simultaneous effect of sex, age and educational level in years was fitted. Apart from sex, coded as a dummy variable, all dependent and independent variables were centred, where centring each variable on its mean corrected for any overlap with the effect of other terms of the model. The multiple regression proved significant (F_2;242) = 9.08; p <0.001, adjusted R_square = 0.45). With regard to regression coefficients, sex and education proved significant (p = 0.002 and p = 0.008 respectively), whereas age did not, due to multicollinearity with education (r_p = 0.422; p < .001) and it was therefore discarded. On average, females obtained lower scores than males (8.71 versus 9.40, sd 1.743 and 1.645 respectively). Education played a positive but modest role, an increase in the score from one education class to the adjacent one accounting approximately half a point (Table 3).

Let y_f,, y_m indicate the score of a female and a male respectively and x the number of years of education. Then, the estimated impact of confounding variables on the TTT Complexity Centred raw scores can be expres e (males coded as 0, females as 1) and centred years of education.

$$(y-\overline{y})={\beta }_1{x}_1+.{\beta }_1(x_1-{\overline{x}}_1)(1$$

The estimation of the linear regression for the raw scores gives:

$$\begin{array}{l}(y_m-9.06)=-.06\times (x-11.35)\\ (y_f-9.06)=.645-.06\times (x-11.35)\end{array}(2$$

Accordingly, adjustment was performed subtracting the estimated contribution of the confounding variables from each raw score, distinctly for females, with x = 1, and for males, with x = 0 in (2). In order to produce the adjustment to be applied to patients raw scores evaluated in rehabilitation practice, Table 4 shows the correction grid with the points to be added to raw Complexity Scores in order to calculate adjusted scores. Once the adjusted distribution had been computed, the identification of a cut-off point that assessed normality or impairment was a crucial step¹⁹. The appropriate criteria were represented by the interval underlying the lowest 5% tail of the adjusted scores in the cumulative distribution.

However, misclassification of performance may arise and needs to be taken into account. In using the widely accepted value of the lower 5% of the normal population (regarded as a reasonable criterion for classifying subjects that are probably not normal) there is an inherent risk of incorrect categorization. The estimation of inferential tolerance limits enable one to obtain the thresholds above (or below) which there is at least (or at most) a desired percentage of the population, and the estimation of these limits keeps errors in performance assessment under control^7,18. With the thirteenth observation, corresponding to the value of 6.25, representing the fifth percentile of the cumulative distribution function, the tenth and the sixteenth observation were identified as the outer and the inner limits, yielding the values of 5.86 and 6.44 respectively. Values equal to, or lower than, the outer tolerance limit indicate a pathological performance, with a controlled error risk. In order to compare the performance in this test to those in other tests, the standardization issue needs to be faced. The commonly used z-scores raise various difficulties, such as an alteration of the statistical dispersion of adjusted scores and problems with floor and ceiling values⁷. Distribution-free techniques are required here, since the best standpoint seems to be that of regarding adjusted scores as raw estimates of performance and hence converting them into an ordinal scale with just a few ordinal values, by means of the cumulative function of adjusted scores. A 5-point scale from (0 to 4), termed equivalent scores, is widely used, where 0 indicates the score that lies below the outer non-parametric tolerance limit of adjusted scores, Equivalent scores 1, 2 and 3 are intermediate between 0 and 4, id est they are obtained in the cumulative adjusted scores distribution. The equivalent score 4 indicates a performance equal to or superior to the median, thus no longer distinguishing between scores found in the upper half of the distribution. Equivalent scores 1, 2 and 3 are intermediate between 0 and 4 on a quasi-interval scale. An equivalent score equal to 0 is considered below the normal range, with a controlled error risk. This contracted scale of equivalent scores is then measured on a quasi-interval scale⁸ and may be viewed as a standardisation of adjusted scores. Table 5 shows the equivalent score limits, the density (i.e. the number of subjects within each equivalent score), and the cumulative frequency of subjects from 0 to 4 equivalent scores.

Discussion

The TTT has proved to be a highly sensitive instrument for the assessment of organization, planning abilities and response flexibility in a less structured context, compared with other neuropsychological tests generally used to evaluate executive functions¹⁶. Previously, the TTT could not be administered during psychometric neuropsychological examinations, owing to the absence of normative values, apt to compare the performance of frontal brain-damaged patients with the mean performance of unimpaired subjects with similar demographic features. The present study has filled this gap, establishing TTT normative data for a wide, healthy population sample, representative of the Italian population (n=256). Statistical analyses were adopted according to the methodology that is most widely used in Italy for the computation of normative values^{2,6–8,14,21,22}. Our findings showed how TTT performance was affected by sex and education. In particular, males performed better than females and the higher the education the higher the TTT scores. Both effects have already been found in other tests for executive functions, such as the Wisconsin card sorting and Weigl tests¹⁴ and show an effect of culture and learning in structuring high-level functions. The relationship with education was also found by Apollonio and collaborators with the FAB². Adjusted scores and inferential cut-off scores were calculated. Moreover, adjusted scores were transformed into equivalent scores, since the availability of equivalent scores makes it possible to evaluate whether a patient presents a homogeneous cognitive profile, or if he/she presents selective deficits in one or more cognitive areas. Therefore, it is now possible to compare the performance of brain-damaged patients directly with the TTT and other neuropsychological tests, using normative data with equivalent scores.

Data availability

F1000Research: Dataset 1. Raw data for ‘Normative data for Lezak’s Tinkertoy test in healthy Italian adults’, Crippa et al. 2016., 10.5256/f1000research.8409.d118684²⁸

Consent

Written informed consent was obtained from each patient in the normative study and from a pathological patient whose construction is depicted.