CAPE for measuring callous-unemotional traits in disadvantaged families: a cross-sectional validation study

Introduction

Callous-unemotional (CU) traits include behaviours that reflect a lack of caring for others and for doing things well, a lack of guilt and remorse, and a lack of emotional depth in interactions with others¹. CU traits across childhood and adolescence have been associated with delinquency, aggression and impairing conduct problems^2–4. These traits have proven useful for designating a subgroup of children with serious conduct problems who show distinct emotional, cognitive, and social correlates relative to other youth with behaviour problems⁵. Further, there is evidence that children with serious behaviour problems who also show elevated levels of CU traits show less positive responses to many interventions used to treat conduct problems^6,7. As a result of this evidence for both their clinical and etiological validity, CU traits have recently been included in the specifier “with Limited Prosocial Emotions” (LPE) for the diagnostic criteria for Conduct Disorder in the DSM 5 (Diagnostic and Statistical Manual of Mental Disorders, 5th Edition;⁸) and for the diagnostic criteria for Oppositional Defiant Disorder and Conduct Disorder in the ICD-11 (International Classification of Diseases version 11;⁹).

Conduct problems are one of the most frequent reasons that youth are referred for mental health treatment¹⁰ and they are very costly to the community due to their association with delinquency¹¹. As noted above, those children with serious conduct problems who also have elevated CU traits show less positive responses to many forms of treatment, possibly because of their deficits in emotional responding to the distress in others and abnormalities in their reward and punishment processing^12–14. However, certain treatments that are tailored to their unique emotional and cognitive style have shown some limited success in children and adolescents with elevated CU traits^7,15,16. Thus, accurate assessment of these traits is critical for treatment planning¹⁷.

With the inclusion in major classification systems used to diagnose childhood behaviour problems and their importance for guiding effective treatment, it is crucial to have measures of CU traits that can be used in high risk samples to make important clinical decisions. This is especially true when children are young, since early interventions tend to be most effective^18–20. To date, much of the research on CU traits has relied on informant rating scales that have proven to reliably assess these traits from the age of three years^16,21 and in a way that is relatively stable between childhood and adolescence^22,23. Further, behaviour ratings provide a very time-efficient method for assessing these traits in research because they are easy to complete and do not require high levels of training to administer and interpret.

However, there are significant limitations in the sole reliance on rating scales for making clinical decisions. First, well-established clinical cut-offs are not available for many of these measures to designate when the level of these traits is severe and impairing enough to warrant a diagnosis^21,24. Second, as is the case for all forms of psychopathology in children²⁵, ratings of CU traits from different informants often show only modest levels of agreement²⁶. This lack of correspondence across different sources of information makes it a challenge for clinicians to know how best to integrate these discrepant reports when making diagnoses²⁷. Third, a number of biases, both intentional (e.g., deception) and unintentional (e.g., social desirability), can influence ratings of CU traits that need to be considered when making clinical decisions and such sources of bias are often difficult to determine from questionnaires^27–29. Fourth, it appears that the optimum assessment of CU traits that detects the construct across all levels of severity requires the use of both positively worded items in which affirmative response denotes more callous and unemotional behaviours (e.g., “do you seem cold and uncaring to others”), as well as negatively worded items in which an affirmative response denotes lower levels of the construct (e.g., “do you feel bad or guilty when you do something wrong”;³⁰). However, negatively worded items can sometimes be difficult to understand and put a strain on a child’s verbal abilities, which are often found to be deficient in children with conduct problems, anyway³¹. Thus, when making clinical decisions, it is important that items are accurately understood by those providing information. A final issue with existing rating scale measures of CU traits is that they were not developed specifically to assess the criteria for the LPE specifier included in the DSM-5 and ICD-11, which further complicates their use for making the diagnosis²¹.

To overcome these limitations of existing measures of CU traits, the Clinical Assessment of Prosocial Emotions, Version 1.1 (CAPE 1.1;³²) was developed to assess CU traits in children and adolescents ages 3 to 21 years of age. The CAPE 1.1 was designed specifically to assess the four symptoms of the LPE specifier now included in the DSM 5: lack of guilt/remorse, callousness and lack of empathy, unconcern about performance, and shallow or deficient affect. The CAPE 1.1 uses the structured professional judgement approach to assessment. That is, very explicit descriptions of the four symptoms are provided and experienced clinicians rate the children on each symptom using a three-point scale (0=not at all descriptive; 1=somewhat descriptive; 2=definitely descriptive) based on all sources of information. The manual has descriptions of prototypical behaviour for each of the four items to aid the interviewer in making their ratings. Also, the CAPE 1.1 is explicitly designed to not rely on any single source of information and it provides a semi-structured interview format to aid the clinician in obtaining information from the child, parents, and other informants. These interviews start with stem questions directly related to the items being rated (e.g., “Does ________ seem to care and be concerned about the feelings of others?”). The clinician follows up by asking for examples to ensure that the informant understood the question and by asking other questions (e.g., “Is this how he is most of the time and with most people”) to aid in scoring. The semi-structured format allows the interviewer to probe and ask for as many examples as needed. The final ratings are only made by considering all sources of information and, to conform to DSM-5 criteria, a diagnosis is made when two or more items are scored as being “very descriptive” (the maximum score of 2) of the child.

In summary, the CAPE 1.1 was designed to overcome many of the limitations of existing measures of CU traits for making clinical diagnoses of the LPE specifier. However, to date, there has been no published data testing the validity of this method. Thus, we investigated the validity of the CAPE 1.1 in a group of socially disadvantaged families that are part of the ‘Troubled Families Scheme’ - implemented by the UK government in 2010. Children who show serious conduct problems tend to come from high-risk backgrounds involving disorganized, unmotivated or disadvantaged families³³, and the scheme provides early intervention for these behaviour problems facilitated by local county councils. The scheme targets families in which parents experience unemployment, their children fail to attend school, and where family members are involved (or at risk of being involved) in drug use and/or other criminal activity. The stated aim is to break the cycle of disadvantage that leads to antisocial behaviour across generations. In the current study, we examined the relations among the CAPE 1.1 item scores and criterion-related measures with well-validated questionnaires of CU traits and psychopathic traits in a sample of families in the Troubled Families Scheme. We also validated the CAPE 1.1 with construct-validity measures of offending, and violence from case file records. We examined the relation between the CAPE 1.1 ratings and well-validated questionnaire measures of externalizing (conduct problems) and internalizing behaviours as well as impact of symptoms on daily living. Finally, we examined the validity of the diagnostic cut-off used by the CAPE 1.1 to determine clinical levels of CU traits, as specified in the DSM-5 criteria for the LPE specifier. We decided not to confound the validation measures with CAPE 1.1 ratings, so ratings were solely based on the semi-structured interview.

Method

Results

Do target children differ in symptoms from their non-target counterparts?

We first tested whether target children for the intervention scored higher on emotional and behaviour problems, as well as CU and psychopathic traits. The mean scores and standard deviations are shown in Figure 1 (see underlying data⁴⁷). Welch t-tests are reported because of violations of normality assumptions. Children targeted for the intervention were higher than their siblings (non-target children) in parent-reported psychopathic traits on the CPTI (t(62.20)= -3.14, p = 0.003, d = -0.72), and this included all subscales: callous/unemotional (t(68.25)= -2.25, p = 0.027, d = -0.51), grandiosity/deception (t(56.50)= -2.96, p = 0.005, d = -0.69), and impulsivity/need for stimulation subscale (t(70.08)= -2.64, p = 0.010, d = -0.60). Target children were higher than non-target children in UNSW CU traits according to both parents and children themselves (t(65.50)= -2.38, p = 0.020, d = -0.54; t(38.83)= -2.61, p = 0.013, d = -0.80) as well as parent-reported externalizing behaviour (t(68.30)= -2.33, p = 0.023, d = -0.53) and they showed more of an adverse impact on their daily living activities from their symptoms based on the SDQ (t(53.19)= -4.11, p = < 0.001, d = -0.96). Thus, target children were similar to their non-target siblings in symptoms of internalizing behaviour problems, regardless of reporter, but were higher on psychopathic traits, CU traits and the impact of their problems on their activities of daily living.

Figure 1. Mean scores (with SD bars) on validation measures by target and non-target children.

(Note: CPTI= Child Problematic Traits Inventory, GD= Grandiosity/deception, CU= Callous/unemotional, INS=Impulsivity/need-for-stimulation, UNSW= University of New South Wales, CR= Child-report, PR= Parent-report).

Clinical assessment ratings of Limited Prosocial Emotions

Figure 2 shows the percentage of target children that scored at each level of severity for each symptom on the CAPE 1.1. The most frequent rating given to children was ‘0’ for all symptoms, ranging from 41% to 59% across the four symptoms. The least frequent rating was the maximum score of “2”, which ranged from 15% to 29% across symptoms. Of the 34 target children, seven children (21%; all males) met diagnostic criteria for the LPE specifier (two or more items were rated at a maximum value of ‘2’). Thus, the majority of the sample did not meet the threshold for the diagnosis, and most did not have symptoms that reached a clinical range.

Figure 2. Proportions of target children given ratings of 0 ‘not descriptive or mildly descriptive’, 1 ‘moderately descriptive, and 2 ‘very descriptive’ on the CAPE 1.1.

To determine whether all items equally discriminated those meeting diagnostic criteria on the CAPE, we examined the frequency of symptom scores within those who did and not meet the threshold for the LPE specifier. The results are provided in Figure 3. For ‘lack of remorse or guilt’, ‘callous-lack of empathy’ and ‘shallow or deficient affect,’ the majority of the seven children who met criteria for the LPE specifier were rated a ‘2’, meaning their behaviour was believed to be very descriptive on these three symptoms. No children meeting the criteria for LPE specifier scored ‘0’ for ‘lack of remorse or guilt’ or ‘callous-lack of empathy’ and only one of these children scored ‘0’ for ‘shallow or deficient affect.’ In contrast, the ratings for ‘unconcerned about performance’ did not seem to differ between children who met the criteria for the LPE specifier and those who did not.

Figure 3. Frequency of use of each CAPE 1.1 rating by CAPE 1.1 item and LPE diagnostic status.

Do CAPE 1.1 ratings of CU traits relate to psychopathic traits, greater risk of violence and offending, and symptoms of internalizing and externalizing behaviour problems?

Figure 4 notes the results of Spearman’s correlations examining the associations of the validation measures (questionnaires and case file records) with CAPE 1.1 ratings (number of symptoms rated 2). An ‘x’ placed across the correlation in the figure denotes that the result is non-significant. Spearman’s rho was used because of the non-parametric nature of many of the measures utilized. CAPE 1.1 scores were significantly correlated with CU traits (when measured by parent report only), psychopathic traits, ratings of violence from their case files, and the negative impact of their mental health symptoms on their daily living. There were no associations found between the CAPE 1.1 and externalizing/internalizing behaviour regardless of reporter, although the relation with externalizing behaviour (as reported by the parent) was moderate.

Figure 4. Spearman correlations among study measures.

(Note: CPTI= Child Problematic Traits Inventory, GD= Grandiosity/deception, CU= Callous/unemotional, INS=Impulsivity/need-for-stimulation, UNSW= University of New South Wales, Child= Child-report, Parent= Parent-report, CAPE= Clinical Assessment of Prosocial Emotions).

Do children who meet the LPE specifier according to the CAPE 1.1 show greater psychopathic traits, greater risk of violence and offending, and symptoms of internalizing and externalizing behaviour problems?

The final test of validity focused on whether the children who met criteria for the LPE specifier (n =7) would be different from those who did not (n = 27) on the various measures of problem behaviour and questionnaire measures of psychopathic traits. The mean scores and standard deviations for the parent and child report measures across the two groups are shown in Figure 5. Levene’s test of equality of variance was nonsignificant except for child reported internalizing behaviour, so we used Student t-tests except in that case. Those children diagnosed with the LPE specifier were found to have higher CU traits according to the UNSW parent report (t(31)= -3.02, p = 0.005, d = -1.29) and higher psychopathic traits (t(32)= -2.32, p = 0.027, d = -0.99) according to the CPTI total – specifically, they were higher on the grandiosity/deception subscale (t(32)= -2.84, p = 0.008, d = -1.20) and the CU subscale (t(32)= -2.32, p = 0.027, d = -0.99). However, the two groups did not differ on the impulsivity/need for stimulation subscale of the CPTI (t(32)= -0.84, p = 0.408, d = -0.36) or any of the other measures. With regard to case file records, children meeting the criteria for the LPE specifier did not differ, from those not meeting criteria, on risk for violence or for contact with young offending services (X² (1, N = 32) = 1.52, p = .217; X² (1, N = 32) = 0.96, p = 0.327, respectively). Of note, no girls met criteria for the LPE specifier.

Figure 5. Mean scores (with SD bars) on validation measures by Limited Prosocial Emotions diagnostic status using the CAPE (Note: CPTI= Child Problematic Traits Inventory, GD= Grandiosity/deception, CU= Callous/unemotional, INS=Impulsivity/need-for-stimulation, UNSW= University of New South Wales, CR= Child-report, PR= Parent-report).

Discussion

In a sample of difficult-to-engage families, we showed that clinician ratings of CU traits using the CAPE 1.1 were associated with parent ratings of CU traits, psychopathic traits, and externalising behaviours. CAPE 1.1 ratings were also associated with risk for violence obtained from case files. These findings provide preliminary evidence for the validity of the CAPE 1.1 as a measure of CU traits, assessed in a way that is consistent with the Limited Prosocial Emotions (LPE) specifier for the diagnosis of Conduct Disorder included in the DSM-5 and ICD-11.

As the CAPE 1.1 and UNSW CU traits measure are both derived from the ASPD and are therefore based on the same historical items, it is possible that they may use similarly worded items. However, as the CPTI was created differently, significant association between the CAPE 1.1 scores and the CU subscale of the CPTI suggests that findings do not solely rely on the validity between similarly worded items.

Using the diagnostic cut-off specified by the CAPE 1.1, we found that 21% of the target children met criteria for the LPE specifier, which is commensurate with estimates from prior research using detained^48,49, clinic-referred⁵⁰, and high-risk⁵¹ samples. When using this cut-off, those who scored above the diagnostic threshold were higher on parent ratings of CU traits and also ratings of grandiosity and deceptiveness, which are part of the interpersonal psychopathic traits. However, this group did not differ on the other measures of externalizing behaviour or risk for violence. There are two possible reasons for this finding. First, the cut-off led to only 7 children in the sample meeting the diagnostic criteria, leading to very low power for detecting group differences. Second, the use of a high-risk sample meant that even those who were not elevated on CU traits likely showed significant problems in adjustment.

The findings provide preliminary evidence for the diagnostic cut-off specified in the CAPE 1.1 manual as an appropriate measure to designate when the level of these traits warrants a diagnosis. CAPE 1.1 ratings using the sum of items scored at the maximum of “2” were significantly correlated with ratings of violence. This signifies that the prototypical presentation of low empathy, lack of guilt, lack of concern and shallow affect was associated with greater problems in adjustment across reporter-based measures but also case file records. Additionally, the greater number of areas where children showed this prototypicality were even more related to maladjustment. Further research with larger samples and with more diversity of problem behaviours is required to determine if the diagnostic cut-off is useful for this tool.

Importantly, the CAPE 1.1 provides a structured method for making clinical decisions, making it potentially useful for clinical settings that require more in-depth assessments than reliance simply on scores from rating scales. Clinical uses will become increasingly common now that CU traits are included in the diagnostic classification systems used globally. Further, as interventions are developed and tested to specifically target the needs of children and adolescents with elevated CU traits⁵², their success will rely on adequate assessment, especially in samples who experience social disadvantage, behaviour problems, poor school attendance, and who show behaviours that are generally difficult to access. Further, in such samples, clinical judgement will be important to ensure that reporters understand the questions and are able to provide information in a way that is appropriate for their cultural and educational background. Specifically, the CAPE 1.1 requires clinicians to gain examples from the informants in their own words, to ensure that the questions are understood and answered in the way that is intended.

Given that the CAPE 1.1 only leads to the rating of four items, it was somewhat surprising that they still formed a relatively internally consistent scale. This finding suggests that scores from the CAPE 1.1 can be used as a continuous measure of CU traits as well. Further, three of the four items differentiated those who scored above the diagnostic criterion from those who did not. Specifically, the frequency at which the item “unconcerned about performance” was rated as being at the symptom level (i.e., a score of 2 or “very descriptive”) was similar for children who did and did not meet the diagnostic criteria for LPE. The small sample leading to a limited number of children meeting criteria for the specifier, again suggests that this finding should be replicated in other samples. However, it does point to the need to evaluate the relative utility of the symptoms used to define the LPE specifier⁵⁰.

These results need to be interpreted in light of several limitations. First, as noted previously, the sample size led to a very small number of children meeting the criteria for the LPE specifier and thus, there was limited power of the tests comparing those meeting and those not meeting the diagnostic threshold for the LPE specifier. The small sample also prevented us from testing potential moderators of the validity of CAPE 1.1. Of particular note, we could not test potential differences that might have been found in the validity of the CAPE 1.1 across age or sex of the child. Second, we only used parent and child reports from semi-structured interviews to score the CAPE 1.1; we did not have access to teachers as potential informants. Also, because we wanted to use behaviour ratings and case files to test the validity of the CAPE 1.1 scores, clinicians were not allowed to use this information in their ratings. Thus, clinicians were not able to use “all sources of available information”, as recommended by the CAPE 1.1 manual³². Third, although the researchers carrying out the CAPE 1.1 had a sufficient level of training to do so (as previously outlined), they would not meet the criteria to make clinical decisions from its use relative to what is recommended in the manual. However, as the CAPE 1.1 was carried out solely for research purposes, rather than diagnostic purposes, that level of training was not necessary. Fourth, because of the training required and the method of obtaining information that relies on collecting multiple sources of information, the CAPE 1.1 is a much more time consuming and expensive method for assessing CU traits when compared to behaviour rating scales. While we have argued that this could be beneficial for many clinical uses, it will be important for future research to test whether the scores from the CAPE 1.1 provide important information over and above that provided by rating scales that would justify the cost.

In sum, the CAPE 1.1 shows promise as a method for assessing CU traits in a way that is a) consistent with the diagnostic criteria for the LPE specifier and b) useful for making complex clinical decisions. Further, this promise was demonstrated in a sample of hard-to-reach families for whom clinical decisions may be difficult to make through other means. As a result, the CAPE 1.1 could provide a method for making important clinical decisions for children and adolescents who are at risk for a particularly severe and chronic pattern of conduct problems, for whom careful treatment planning is essential.

Data availability