In the last decades, many countries have installed early prevention programs for young children to mitigate early psycho-social risks (Manning et al., 2010; Benz & Sidor, 2013). Empirical evidence has repeatedly shown that adversities like poverty and maltreatment as well as biological risks like genetic predisposition for emotional dysregulation increase the risk for early mental health problems in children if protective factors or family support systems do not buffer against such stressors (Demeusy et al., 2018; Masten & Monn, 2015). Detecting early onset mental health symptoms in young children is a prerequisite for timely intervention as early mental health problems and their emotional precursors (e.g. emotional dysregulation) can persist during childhood (Bufferd et al., 2012; Shaw et al., 2001; Mathiesen & Sanson, 2000) and may be precursors of later emotional or social problems (Briggs-Gowan & Carter, 2008; Mathiesen & Sanson, 2000; McDermott et al., 2013).

Mental health problems can be validly diagnosed even in very young children (Egger & Angold, 2006; Zeanah & Lieberman, 2016; Mulrooney et al., 2019). International studies report prevalence rates for early childhood mental health problems range from 5 to 26% with some variation by assessment approach, the specific diagnoses included, the sample characteristics, the country, the percentage of children at risk, and the age range studied. The most rigorous studies report rates of 10–20% in the general population (Egger & Angold, 2006; Brauner & Stephens, 2006; Wichstrøm et al., 2012) and even in children as young as 1.5 years, mental health problems were reported in 16–18% (Skovgaard et al., 2007). In addition, early signs of emotional dysregulation have been identified as important precursors of later mental health problems in childhood (NICHD Early Childhood Research Network, 2004; Holtmann et al., 2011; Deutz et al., 2018). Therefore, early difficulties in emotion regulation or self-control are important precursors and correlates of mental health problems in early childhood.

Early screening for mental health problems or symptoms could help identifying young children with needs for further systematic evaluation of their problems and preventing maladaptive developmental pathways (Weitzman & Wegner, 2015; Gleason et al., 2007). However, many young children with emotional and behavioral problems are not identified in primary care and most receive no treatment (Weithase et al., 2017). One possible explanation is the lack of a standardized, validated, and reliable screening assessment meeting the needs of pediatric primary care or other early childhood care units with often limited time for assessment (Fallucco et al., 2017). This is also the case in German speaking countries. Thus, we examined such a standardized tool, the German version of the ECSA.

Mental Health Screening in Early Childhood

Mental health screenings by community practitioners show a comparable but not identical detection rate of mental health problems in young children as diagnostic tools used by experts (Charach et al., 2020). There is a wide array of mental health assessment approaches for young children (Szaniecki & Barnes, 2016), but not all of them are available in German. Therefore, there is a need to provide a screening tool for German speaking countries that is valid and efficient for early childhood. Although many of the symptoms of an early childhood mental health assessment are also included in the assessment of older children, some clinical domains require specific developmental adjustments for early childhood (Gleason et al., 2010).

The Early Childhood Screening Assessment (ECSA; Gleason et al., 2010)Footnote 1 identifies young children who are at risk for mental health problems. It is well validated in the United States in comparison to other clinical screenings (e.g. CBCL; Achenbach & Rescorla, 2000) and clinical interviews, has a published and validated clinical cut-off score, and has been used in the United States to assess clinical risk status of young children (Gleason et al., 2010; Fallucco et al., 2017). Using the Diagnostic Interview for the Preschool Age (DIPA; Scheeringa & Haslett, 2010) as a gold standard diagnostic tool, the ECSA was 86% sensitive and 83% specific in identifying clinical diagnosis. Sensitivity here is the rate of correctly identified children with a clinical diagnosis (i.e., true positives). Specificity here is the rate of correctly classified children without clinical diagnosis (i.e., true negatives). Therefore, the ECSA is a promising, short, easy to understand, validated tool, and easily applied in settings with time constraints, like primary care practice (Fallucco et al., 2017).

In the United States, the ECSA is one of the screening instruments for emotional and behavioral problems included in the American Academy of Pediatrics’ (AAP) Screening Technical Assistance and Resource Center (AAP (2017) Screening Time. https://screeningtime.org/star-center/#/screening-tools accessed November 17, 2021). Moreover, for early childhood, the ECSA is the only measure validated against a formal psychiatric diagnosis and is included in the tool kit (Fallucco et al., 2017). Meanwhile, a Spanish version of the ECSA has been published also showing good sensitivity and specificity compared to the CBCL (Cano et al., 2018) and a German version has been used in a study on early adversity and child development (Zimmermann et al., 2016).

Internalizing and Externalizing Symptoms in Early Childhood

In the original study, Gleason et al. (2010) reported the results of an exploratory principal component analysis of the ECSA, which yielded one primary factor including negative mood, anxiety, disruptive behavior, and attention problems with no distinction of externalizing and internalizing symptom factors. However, there is a long scientific debate whether several mental health problems can be differentiated already in young children or whether they overlap as signs of one general emotional and behavioral dysregulation at least in parent report (Egger & Angold, 2006; Kerr et al., 2007; Strickland et al., 2011). Many empirical studies favor the distinction of internalizing and externalizing mental health problems already in early childhood and research on the CBCL repeatedly supports this distinction already in two- and three-year-olds (Achenbach et al., 1987; Koot et al., 1997). As previous research suggests a differentiation of internalizing and externalizing symptoms at that early age, we wanted to examine whether the factor structure of the ECSA favors the published one-factor solution or whether a two-factor solution would better explain the data when using a confirmatory factor analysis. Moreover, we intended to replicate and extend previous findings on the concordance, diagnostic accuracy, and correlation with an accepted gold standard, the CBCL as reported in previous studies (Gleason et al., 2010; Cano et al., 2018). The comparison with the CBCL can provide evidence for convergent validity when both measures show high positive correlations.

Emotional Dysregulation and Early Mental Health Problems

Besides studying the validity of the German ECSA in comparison to other clinical screening tools like the CBCL, we intended to add other indices of validity by examining the associations with early signs of emotional dysregulation or (mal-)adaptive self-regulation like ego-undercontrol, ego-resiliency, and field-independence (Block & Block, 1980; Eisenberg et al., 2010). Early emotional dysregulation is a developmental precursor of later signs of mental health problems (NICHD, 2004; Olson et al., 2005; Deutz et al., 2018; Holtmann et al., 2011), and ego-undercontrol, ego-resiliency, and field-independence reflect differences in children’s ability to regulate emotions and needs in social contexts. Ego-undercontrol characterizes the impaired ability to control and delay one’s needs, impulses, or emotions. Ego-resiliency describes the ability to modulate one’s emotions and needs appropriately depending on the situation and to adapt quickly to stress (Block & Block, 1980; Eisenberg et al., 2010). The third dimension provided by Block & Block (1980), field independence reflects the ability to behave in a self-determined manner within a social context (Lamb et al., 1988). Low ego-resiliency and high ego-undercontrol characterize children’s emotional dysregulation and both are concurrently and longitudinally associated with maladjustment (Eisenberg et al., 2010; Juffer et al., 2004; Deutz et al., 2018). Based on earlier research, we expected ego-undercontrol to be positively associated with externalizing symptoms, whereas ego-resiliency and field-independence should be negatively associated with both externalizing and internalizing symptoms (Taylor et al., 2013; Eisenberg et al., 2010; Martel et al., 2009). Therefore, we examined the validity of the ECSA in its relation to ego-resiliency, ego-undercontrol, and field-independence using the California Child Q-sort (CCQ) as well as the symptom specific CCQ scales aggressiveness and anxiety. We expect that CCQ aggression will correlate positively with the ECSA externalizing scale and CCQ anxiety positively with the ECSA internalizing scale (Asendorpf et al., 2008).

Caregiver Depression and Early Mental Health Problems

The ECSA also includes items to screen for signs of depression in caregivers. Maternal depression is a major risk factor for child mental health problems (Goodman et al., 2011). The impact of caregiver depression may be due to lower caregiving sensitivity, impaired caregiver-child interactions or caregiving difficulties (Bernard et al., 2018; Newland et al., 2016). Thus, early screening not only of children’s mental health problems but also of caregiver depression with one screening tool is an efficient way to offer support for parents of young children allowing fast assistance for caregivers (Lind et al., 2017). Moreover, caregiver depression may also influence the caregiver’s perception of the own child as challenging or particularly difficult (Gleason et al., 2010; Müller & Furniss, 2013). As the ECSA is a parent report tool, we examined whether the ECSA screening of caregiver depression and children’s mental health problems are also associated in the German ECSA adding further indicators for the validity of the ECSA and showing the usefulness of the ECSA in application in primary child care settings.

Aims of the Study

The objectives of this study were to evaluate the German version of the Early Childhood Screening Assessment (ECSA) by (1) examining the factor structure of the German ECSA and the internal consistency, (2) investigating concurrent validity evidence by evaluating the associations with CBCL scores, (3) examining the accuracy of clinical at risk domains by assessing sensitivity, specificity, and clinical cut-off scores in a German sample in comparison to the respective CBCL cut-off criteria, (4) further demonstrating concurrent validity by examining associations of ECSA scores with the regulatory personality characteristics ego-resiliency, ego-undercontrol, field-independence, and in addition with anxiety and aggression as specific internalizing and externalizing indicators of mental health problems, and (5) examining associations between ECSA scores and caregiver depression.

Methods

Sample and Procedure

The sample consisted of 282 German children between 17 and 73 months (mean age = 43.63 months; SD = 11.45 months), 46% of them female. 99.3% of the children were 60 months or younger. Ten children were younger than 24 months, one child was 5 years and 2 months, and one child with developmental delays was 73 months old. The first validation of the ECSA included children 18–60 months in the United States. Similar to other studies on psychometric properties of early childhood mental health screening tools (i.e., CBCL 1 ½–5) we slightly extended the age range of children included in our study (Rugytė et al., 2021; Rescorla et al., 2019). This further assesses the measures functioning.

The sample includes two groups: a risk group (N = 41; mean age = 42.46 months; SD = 14.13 months; 36.6% female) and a non-risk group (N = 241; mean age = 43.95 months; SD = 10.83 months; 48% female). Participants of the risk group came from early intervention centers and foster care services, participants of the non-risk group were recruited in childcare centers in Germany by means of public announcement or direct contact to institutions. Biological parents or foster parents were informed about the study aims and we obtained informed consent. All caregivers answered the two symptom checklists, the ECSA and the CBCL. In addition, 42 caregivers provided personality descriptions of their children using the California Child Q-Sort (Block & Block, 1980).

Instruments

Mental Health Problems

Early Childhood Screening Assessment (ECSA)

Young children’s mental health problems were assessed by use of the Early Childhood Screening Assessment (ECSA; Gleason et al., 2010). A German version was developed for this study (SFK; “Screening Frühe Kindheit”, Zimmermann, 2013). The ECSA items were translated by the first author and translated back into English by collaborators. The ECSA is a 36-item, parent report measure on young children’s mental health symptoms. Child symptom items focus on internalizing and externalizing symptoms, regulatory processes, or interpersonal relationship patterns, and are rated on a Likert scale from 0 (never/rarely) to 2 (always/almost always). A total mental health problem score is the sum score of all 36 items.

The ECSA also includes four caregiver distress and depression items including items that are the validated Unites States Preventative Health Task Force (USPHTF) depression questions (Fallucco et al., 2020). In this study, the ECSA caregiver depression scale showed an internal consistency of α = 0.76, comparable to earlier research (Gleason et al., 2010). The ECSA demonstrated strong convergent and criterion validity, high internal consistency, and test-retest reliability (Gleason et al., 2010).

Child Behavior Checklist (CBCL)

The preschool Child Behavior Checklist (CBCL 1.1/2 –5; Achenbach & Rescorla, 2000) consists of 99 items concerning the prevalence of problem behavior during the last two months. Items are scored on a three-point Likert scale (0 = “not true”, 1 = “sometimes/somewhat true”, 2 = “exactly/often true”), with higher scores indicating higher levels of behavior problems. The CBCL discriminates seven syndrome scales. The syndrome scales “Emotionally Reactive” (α = 0.69), “Anxious/ Depressed” (α = 0.65), “Somatic Complaints” (α = 0.56), and “Withdrawn” (α = 0.62) can be aggregated to the Internalizing Problems Scale (α = 0.84). The Externalizing Problems Scale (α = 0.88) consists of the “Attention Problems” (α = 0.60) and “Aggressive Behavior” (α = 0.87) syndrome scales (Rescorla et al., 2011). Internal consistencies assessed in this study using Cronbach’s alpha were high, for the internalizing problems scale (α = 0.82), the externalizing problems scale (α = 0.89), and the total problems scale (α = 0.93), comparable to earlier research (Arbeitsgruppe Deutsche Child Behavior Checklist, 2000). We used the German reliable and validated version of the CBCL in this study (Arbeitsgruppe Deutsche Child Behavior Checklist, 2000).

Personality Characteristics

California Child Q-sort (CCQ)

The California Child Q-sort (CCQ; Block & Block, 1980) consists of 100 items describing a wide range of children’s characteristics including behavioral, emotional, cognitive, and social traits. Parents described their child by sorting all CCQ-items into nine categories from 1 “least characteristic” to 9 “most characteristic” in a fixed standardized distribution (see Block & Block, 1980). Children’s individual Q-sort descriptions then are correlated with the five Q-sort prototypes ego-resiliency, ego-undercontrol, field-independence (Block & Block, 1980), aggressiveness, and anxiety (Zimmermann et al., 2009; Zimmermann & Scheuerer-Englisch, 2013). The resulting five correlation coefficients per child are scores representing each subject’s similarity with these prototypes (i.e., prototypicity) ranging from −1.00 (i.e. highly dissimilar. e.g., low ego resiliency) to +1.00 (i.e. highly similar, e.g. high ego resiliency). The CCQ is a valid instrument for early childhood up to early adolescence (Block & Block, 1980) and the derived variables are robust indicators and predictors of children’s general trait self-regulation and predictive of later (mal-)adjustment, mental health problems, and personality (Taylor et al., 2013; Eisenberg et al., 2010; Asendorpf et al., 2008; Martel et al., 2009; Syed et al., 2020).

The prototypic Q-sort for ego-resiliency describes the ability to control one’s emotions appropriately to the situation especially under stress. The prototypic Q-sort for ego-undercontrol characterizes a child that is low in delay of gratification and shows a predisposition to express impulses immediately. The field-independence prototype describes a child, which remains self-determined even within social contexts and resists peer pressure (Block & Block, 1980). The CCQ aggressiveness prototype characterizes children with hostility and unregulated intense anger, whereas the anxiety prototype describes children who show social withdrawal and intense fear (Zimmermann et al., 2009).

Results

In a first step, we examined whether the reported one-factor structure of the English ECSA (Gleason et al., 2010) also explains the data appropriately for the German ECSA. Therefore, we conducted a Confirmatory Factor Analysis (CFA) in Mplus with MLM estimation of parameters to test the underlying factorial structure of the ECSA. MLM was used as an estimator because of its robustness to non-normality (Muthén & Muthén, 2012). Due to the otherwise high number of indicators, the items were aggregated into item parcels which served as indicators in the models (Little et al., 2002). The parcels were means of three to seven items. Because the ECSA was developed with the goal to identify global mental health problems including internalizing and externalizing behaviors, the items were assigned to groups of parcels that differed by content: Three parcels included items that reflect internalizing behavior, three parcels with items indicating externalizing behavior, and one parcel consisting of the remaining items (e.g. eating and sleep problems). The externalizing and internalizing items were randomly assigned to the externalizing and internalizing parcels respectively. Item parceling as well as the following analysis were repeated two times to examine the stability of the results with respect to different assignments of items to parcels.

Confirmatory Factor Analysis

First, a one-factor model was tested for the ECSA as proposed for the original version of the ECSA (Gleason et al., 2010). As presented in Table 1, fit statistics as well as standardized factor loadings for the three randomly assigned sets of parcels show comparable results for the three CFAs. This indicates that the results are not affected by the assignment of items to parcels. However, the fit statistics indicate that the one-factor model did not fit the data well using RMSEA < 0.06 and CFI < 0.95 as standard thresholds (Muthén & Muthén, 2012).

Table 1 Fit-Indices and standardized factor loadings from confirmatory factor analysis of the ECSA: One-factor model and two-factor model for externalizing and internalizing items
Full size table

Thus, we assumed that at least two different factors could underlie the ECSA differentiating externalizing and internalizing behaviors. Therefore, we tested a two-factor model for the same six parcels indicating internalizing and externalizing symptoms while the unclassified items were excluded in this model. The correlations between the factors were freely estimated. The results for the two-factor model are also presented in Table 1. Again, the results are stable for the three different sets of parcels. The two factors are substantially correlated with estimates ranging from 0.59 to 0.63. The two-factor model fitted the data well. A model test comparing the two-factor model with a nested one-factor model (i.e., a one-factor model with three externalizing and three internalizing parcels as indicators) using MLM as estimator and the Satorra-Bentler scaled chi-square difference test (Satorra & Bentler, 2010) showed that the two-factor model fitted the data significantly better than the nested one-factor model (scaled \(\chi _{diff}^2\) = 51.39, df = 1, p < 0.001). Thus, the results indicate that it is reasonable to distinguish between internalizing and externalizing symptoms in the German ECSA.

Descriptive Statistics of the ECSA 2-Factor Solution

Scale and item statistics for the ECSA are summarized in Table 2. The descriptive statistics indicate high skewness and kurtosis for the ECSA scales in this mainly non-clinical sample. Means and ranges of part-whole corrected item-total correlations lie within an acceptable range. Cronbach’s α for children’s mental health problems are presented in the last column of Table 2 showing good and robust reliability estimates for the total problem score as well as the externalizing scale (α = 0.88). Reliability for the internalizing scale is somehow lower with α = 0.70, but still at an acceptable level. The externalizing scale consists of items 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 19, 23, 24, 25, 26, 27, 28, 31, 32, and 34. The internalizing scale consist of items 1, 2, 4, 12, 16, 17, 18, 20, 30, and 33.

Table 2 ECSA item statistics, scale statistics, and reliability estimates
Full size table

Convergent Validity Evidence

In order to examine the validity of the German ECSA for assessing mental health problems in young children, correlations between the ECSA and CBCL were calculated (Table 3). The overall scores on general mental health problems of the ECSA and the CBCL correlated significantly and highly positively. Moreover, the respective internalizing and externalizing scales of the ECSA and the CBCL showed significant positive associations with each other and also correlated significantly higher than the two cross-symptom correlations of externalizing and internalizing symptoms (z = 4.4, p < 0.001 for ECSA internalizing and CBCL externalizing; z = −0.6.7, p < 0.001 for ECSA externalizing internalizing and CBCL internalizing). Age was not associated with scores on the ECSA and CBCL scales. However, boys showed significantly higher scores for the ECSA total problem scale (p = 0.043) and the externalizing scale (p = 0.023). Controlling correlations for gender did not change patterns presented in Table 3.

Table 3 Correlations between ECSA and CBCL scales
Full size table

Criterion-Related Validity Evidence

Next, we analyzed the German ECSA’s association with clinical or at-risk results on the CBCL in two analyses. As the ECSA is a screening tool that should be used to identify children with a risk for mental health problems, we used the sub-clinical CBCL cutoff (T ≥ 60) for this comparison, which includes both clinical and borderline cases (Achenbach & Rescorla, 2000). We then examined, whether the risk group and the non-risk group of this sample significantly differ in ECSA scores.

Classification Accuracy Compared to the CBCL

A Receiver Operating Characteristics (ROC) analysis was conducted using the R package OptimalCutpoint (López-Ratón et al., 2014) in order to (1) determine an adequate cut-off score for the ECSA scales and (2) evaluate the concurrent validity using the ECSA (see Fawcett, 2006). Optimal cut-off scores for the three ECSA scales (i.e. total, internalizing, and externalizing) were identified using borderline cut-offs on the corresponding scales of the CBCL as the criteria. In Fig. 1 the ROC – illustrating the tradeoffs between benefits (i.e. the true positive rate) and costs (i.e. false positive rate) (Fawcett, 2006) – for the three ECSA scales are presented. In order to identify an adequate cut-off score for the ECSA the Youden’s Index was used as a criterion because it considers the relative costs of the two types of errors (false positives and false negatives) as approximately equally important (Le, 2006).

Fig. 1
figure 1

Receiver Operating Characteristic (ROC) for the ECSA Total Problem Scale, the Externalizing Scale and the Internalizing Scale With the Respective Criterion CBCL Classification (T-values ≥ 60)

Full size image

Next, we analyzed the associations of the ECSA and the CBCL classifications of children falling in the clinical and borderline groups based on the ROC analysis presented above for the total problem scores, the internalizing, and the externalizing scales, respectively. Table 4 presents the concordance along with sensitivity, specificity, the predictive values, and the area under the curve as indices of the accuracy of the ECSA classification of children at risk compared to the respective CBCL classification are (see Fawcett, 2006). Result show that the ECSA total problem scale classification of children at risk identified 17 of 18 children of clinical and borderline cases as classified by the CBCL cut-off (T ≥ 60) when applying the ECSA cut-off of 22. This equals a high sensitivity of 94.4% and a good specificity of 89.4% with 236 of 264 children identically classified as non-risk children. The AUC of 0.94 shows that the calculated cut-off leads to a high diagnostic accuracy (Swets, 1988).

Table 4 Association between ECSA and CBCL total scores, internalizing and externalizing scales and indicators of classification accuracy
Full size table

Concordance, sensitivity, and specificity for the ECSA internalizing and externalizing classification are also presented in Table 4. The quality indices for the optimal ECSA internalizing cut-off score were slightly lower as indicated by the flatter ROC curve in Fig. 1. Using a cut-off score of 3 for classifying the internalizing group, the ECSA identified 85% of the children with a CBCL internalizing symptoms classification (T ≥ 60). The specificity of the ECSA internalizing grouping is 66% for identifying children beyond the borderline CBCL internalizing scale cut-off with an AUC of 0.85. This shows that the cut-off leads to a good accuracy falling in the moderate region but close to the AUC threshold of 0.90 (Swets, 1988). Psychometric properties for the ECSA externalizing scale were good. The obtained cut-off score of 15 yielded a sensitivity of 92% and specificity of 90% when compared with the cut-off score for externalizing problems of the CBCL (T ≥ 60).

Additional Convergent Validity Evidence

Finally, we examined whether the ECSA scales were associated with indicators of self-regulation assessed by the CCQ. As shown in Table 5, ego-resiliency and field-independence were significantly negatively associated with all symptom scales of the ECSA. Thus, higher scores on the symptom scales (internalizing and externalizing) were associated with lower ego-resiliency and field-independence scores. As expected, ego-undercontrol and aggressiveness were significantly positively associated with the overall symptom scale of the ECSA and all externalizing subscales. Thus, children characterized as aggressive and ego-undercontrolled in the CCQ were described as more externalizing and as having more total mental health problems in the ECSA. Likewise, anxiety on the CCQ was significantly positively associated with internalizing symptoms but not externalizing symptoms.

Table 5 Correlations between ECSA scales and CCQ prototypicity
Full size table

Overall, the associations between the three ECSA scales and the CBCL on the one hand and the CCQ on the other hand, provide evidence for convergent validity of the ECSA.

Criterion-Related Validity Evidence

Comparison of Risk Group and Non-Risk Group

As ECSA and CBCL are screening tools for mental health problems, we finally looked at criterion-related validity evidence for both measures examining mean differences between the risk group and the non-risk group. As expected, the risk group (M = 17.97, SD = 10.61) received significantly higher scores on the ECSA total scale than the non-risk group (M = 11.79, SD = 8.57), t(280) = −4.11, p < 0.01 (Glass’s Δ = 0.72 given the non-risk group SD). Similarly, the risk group also obtained higher scores on the CBCL (M = 50.54, SD = 10.04) than the non-risk group (M = 44.34, SD = 9.01), t(280) = −4.01, p < 0.01. The difference amounts 0.69 standard deviations using the control groups SD, i.e. Glass’s Δ = 0.69. Thus, the ECSA differentiates children from risk and non-risk groups for mental health problems similarly to the CBCL.

ECSA Scores and Caregiver Depression

Finally, caregiver depressive symptoms assessed with the ECSA parent items correlated significantly with children’s total problems scale, internalizing problems, and externalizing problems (r(279) = 0.44; r(279) = 0.28; r(279) = 0.41; all p < 0.0001, respectively). Thus, parents with higher depressive symptoms report that their children show more mental health problems including internalizing and externalizing symptoms.

Discussion

The primary purpose of this study was to examine the psychometric properties of the German version of the Early Child Screening Assessment in young children and to evaluate whether we can recommend it for use in pediatric and child mental health units in German speaking countries. First, we investigated whether a one-factor model or a two-factor model differentiating externalizing and internalizing symptoms would better fit the data for this age group. A differentiation between the two mental health domains has been reported in many other studies on early childhood mental health problems (Olino et al., 2014; Koot et al., 1997).

The findings indeed support a two-factor solution, separating externalizing and internalizing symptom scales. The confirmatory factor analysis for two factors shows a good fit to the data and the internal consistencies of both scales are good; somehow better for externalizing symptoms compared to internalizing symptoms. Thus, comparable to the distinction found with the well-established CBCL in that age group (Koot et al., 1997) we suggest to differentiate externalizing and internalizing symptoms when using the German ECSA. This finding is in contrast to other studies on the psychometric properties of the ECSA which report a one-factor model based on exploratory approaches, like principal component analysis. However, the CFA used in this study considers the skewed distribution of the ECSA items and the results support differentiating an externalizing and an internalizing factor in the ECSA. As the sample of this study (mean age 43.63 months) is slightly older than the original ECSA sample (mean age 35.5 months) as reported by Gleason et al. (2010) internalizing and externalizing mental health problems might already be more evident. However, the scale specific associations with the two respective CBCL dimensions also supports the differentiation of internalizing and externalizing ECSA dimensions. We not only found significant positive associations between the ECSA and the CBCL total scores but also specific significant associations of the respective externalizing and internalizing scores of both measures. Thus, ECSA results in quite similar mental health scores in early childhood as the CBCL. This clearly supports the convergent validity of the ECSA and corroborates results reported in the United States (Gleason et al., 2010) and Spain (Cano et al., 2018) for the total scale.

Next, we examined the ECSA’s accuracy of identifying clinical and borderline cases on the CBCL. Neither measure provides a clinical diagnosis, but are clinical screening tools. Therefore, studying concordance including CBCL borderline cases can improve sensitivity. The sensitivity and specificity of the ECSA total score are good and comparable to those reported for the English and the Spanish version (Gleason et al., 2010; Cano et al., 2018). We found a high accuracy in identifying children with externalizing symptoms compared to the CBCL. The ECSA internalizing scale also showed good sensitivity, which is highly relevant for a screening tool but only moderate specificity compared to the CBCL internalizing scale. This may be due to the low ECSA cut-off derived from the ROC analysis. One possible reason for the moderate specificity in comparison to children classified as children with internalizing symptoms based on the CBCL cutoff scores might be the absence of items assessing children’s somatic complaints in the ECSA. Older children score higher on the Somatic Complaints Syndrome Scale of the CBCL (Achenbach & Rescorla, 2000) and somatic symptoms are less salient in early childhood. The ECSA internalizing items include many emotional symptoms. However, the CBCL showed lower sensitivity and specificity than the ECSA when compared to clinical diagnoses based on clinical interviews (Gleason et al., 2010). Thus, future research should again compare ECSA scores with diagnoses derived from clinical child interviews that also include context and detailed information (Gleason et al., 2010).

A topic for future research will be the clinical cut-off. Based on the clinical interviews, Gleason and colleagues (2010) report an ECSA cut-off for mental health problems for the United States for the total problem score of 18. The clinical cut-off score for the German ECSA derived in the ROC analyses in comparison to the CBCL is 22 and somewhat higher. Cano et al. (2018) report an even higher cut-off of 24 for the Spanish ECSA, also compared to the CBCL cut-off of T ≥ 60, as we did in this study. Thus, differences in clinical cut-off scores may depend on the clinical diagnostic tool or sample differences in symptom severity. However, differences between countries may also be the result on different cultural norms. There are similar cross-cultural differences in cut off scores for the SDQ (Goodman, 2001; Janitza et al., 2020) or the CBCL (Rescorla et al., 2014). Thus, future studies need to investigate measurement invariance of ECSA scales between different countries. Beside the convergence with the CBCL, the results also showed that the ECSA scores differentiate children of the risk group of this study from the non-risk group children. We conclude that this also supports convergent and discriminant validity of the ECSA and the use of the ECSA in the applied field.

Moreover, the study provided additional validity evidence of the ECSA by examining associations of ECSA scores with indicators of children’s early emotional dysregulation (Block & Block, 1980; Deutz et al., 2018). Many mental health problems are associated with problems in regulating emotions and impulses (Eisenberg et al., 2010). Block’s concepts of ego-undercontrol and ego-resiliency tap into such self-regulatory characteristics that already can be found in early childhood (Wessels et al., 1997) and that are related to mental health problems (Huey & Weisz, 1997; Martel et al., 2009). The study shows that young children with high ECSA total scores are characterized as emotionally and behaviorally dysregulated with low ego-resiliency, low field-independence, and high ego-undercontrol, aggressiveness, and anxiety. In addition, the negative associations between ECSA internalizing and externalizing problem scores with ego-resiliency support the assumption that mental health problems also are a sign of early emotional dysregulation. This is also in line with results by Deutz et al. (2018) who reported that children showing the dysregulatory profile in the SDQ have low ego-resiliency scores and studies showing a longitudinal prediction of later internalizing and externalizing problems from low ego-resiliency (Winsper et al., 2019). The associations between the CCQ variables and the ECSA internalizing and externalizing scores suggest first signs of discriminant validity of the two-factor solution of the ECSA as externalizing symptoms are clearly associated with ego-undercontrol and aggressiveness, whereas internalizing symptoms are clearly related to CCQ anxiety.

Finally, we replicated earlier findings, showing that caregiver depression is associated with higher mental health problems in young children (Fallucco et al., 2020; O’Connor et al., 2016). This result is relevant for practitioners in two respects. First, the replicated finding that caregiver depression is a risk factor for children’s mental health problems can guide intervention to focus on both the child and the parent. Interestingly, we found similar associations for children’s internalizing behaviors as reported in the meta-analyses by Goodman et al. (2011) but clearly higher associations for externalizing behaviors. It may well be that this short ECSA parent depression screening is especially valid for parents who experience their child as oppositional and difficult to control. Second, practitioners should consider that caregiver’s ratings of their children’s mental health can be biased by the caregiver’s own mental health problems (Müller & Furniss, 2013) whereas the child can show less symptoms when observed directly. Thus, using the ECSA as a screening for both caregiver’s and children’s mental health problems at the same time may well improve identifying highly vulnerable parent-child dyads that urgently need external assistance.

Although the study shows results that are consistent with earlier research, it also has limitations. The sample size is appropriate but more children at risk need to be screened to generalize the results. Given the importance of including a diverse range of children and families in a normative sample, these findings should be considered a first step and should be followed by studies that oversample children with established risks or conditions and/or from families from a wide range of demographic and socioeconomic circumstances, including poverty and limited access to care. In addition, we did not examine concordance with a clinical child interview or with observation of children’s mental health problems.

Overall, the study shows that the German version of the Early Childhood Screening Assessment, the “Screening Frühe Kindheit” (SFK), is a reliable and valid instrument to screen for mental health problems in a diverse group of children with elevated risk of mental health concerns. Compared to the CBCL, this screening instrument is shorter and more time-efficient with comparable validity. Thus, pediatric care, day care, or early childhood prevention programs can use it easily for identifying children at risk or for evaluating intervention effects. Future research with the ECSA might examine measurement invariance between countries and examine a two-factor solution in more detail.