Behavioural/cognitive‐behavioural group‐based parenting interventions for children age 3‐12 with early onset conduct problems
Abstract
This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:
To examine the effectiveness of behavioural/cognitive‐behavioural group‐based parenting interventions for children with early onset conduct problems in improving a) child behaviour outcomes and b) parenting skills and parental mental health.
To critically appraise and summarise current evidence on the incremental resource use, costs and cost‐effectiveness of behavioural/cognitive‐behavioural group‐based parenting interventions when compared to treatment as usual.
Background
Description of the condition
Conduct problems in children are common and costly. Approximately 5 to 10% of UK and US children aged 5 to 15 years present clinically significant conduct problems (Offord 1989; Loeber 2001; Task Force 2006). Conduct problems are the most common reason for referral to psychological and psychiatric services in childhood (Hutchings 2004) and typically include: troublesome, disruptive and aggressive behaviour; an unwillingness/inability to perform school work; few positive interactions with adults; poor social skills; low self‐esteem; non‐compliance with instructions; and emotional volatility (Loeber 2000; Scottish Executive 2001; Task Force 2006). Conduct problems exist on a continuum of severity (Burke 2002; Dretzke 2009), and children with the most severe disruptive behaviours may be diagnosed with Conduct Disorder (CD) or Oppositional Defiant Disorder (ODD) (see Diagnostic and Statistical Manual of Mental Disorders IV (DSM‐IV 2000)). Less severe conduct problems, if left untreated, can develop over time into CD. Conduct problems are three to four times more likely to be present in boys than girls (Burke 2002) and are sometimes co‐morbid with Attention Deficit Disorder (ADD)/Attention Deficit Hyperactivity Disorder (ADHD) (Loeber 2000; Burke 2002).
The prognosis for early‐onset conduct problems (when compared with adolescent onset) is poor and the negative outcomes in adolescence and adulthood may include: antisocial and criminal behaviour (Carey 2000; Dretzke 2009); psychiatric disorders; drug and alcohol abuse; higher rates of hospitalisation and mortality; higher rates of school drop‐out and lower levels of educational attainment; greater unemployment; family breakdown; and intergenerational transmission of conduct problems to children (Moffitt 1993; Loeber 2000; Burke 2002; Broidy 2003; Farrington 2007). In Western countries, the prevalence of conduct problems has increased five fold between 1930 and 2000 (Robins 1999).
The costs of early onset conduct problems to society are considerable. Compared with children without conduct problems, children with severe conduct problems are more likely to require remedial help at primary and secondary school; are up to 10 times more likely to leave school with no educational or vocational qualifications; will make significantly more use of primary care services (e.g. doctor, hospital, speech therapist) (Edwards 2007; McGroder 2009) and are significantly more likely to have contact with the police in adolescence (Gregg 1999). By the age of 28, the utilisation of health, social, education and legal services may be 10 times higher for individuals with a clinical diagnosis of conduct disorder at age 10 (EUR 104,416; GBP 70,019; USD 137,450), when compared to those with no problems (EUR 11,069; GBP 7,423; USD 14,571); the costs for those with non‐clinical conduct problems at age 10 (i.e. who do not meet the diagnostic criteria) have also been found to be 3.5 times higher (EUR 38,836; GBP 35,311; USD 57,311) (Scott 2001; Fergusson 2005).
There is considerable evidence that poor quality parenting is one of the most important precursors of early onset conduct problems (Lipsey 1998; Farrington 2007; Odgers 2008). Inadequate parenting is typically characterised by ineffective parenting skills, including low levels of parental supervision and involvement, and punitive and inconsistent discipline. These tend to positively reinforce childhood aggressive behaviour and extinguish positive behaviours by not giving them attention (Farrington 1999; Patterson 2002; Reid 2002). Moreover, other findings indicate that parental distress and mental illness, substance abuse and disrupted family life (all of which can affect the quality of parent‐child interactions), are involved in the aetiology of early onset conduct problems (Mash 1983; Shaw 1994; Hogan 2002). However, the causal link between parental stress and depression and childhood problem behaviour may be bi‐directional, such that parents and children reciprocally affect and shape one another’s behaviour (Patterson 2002; Long 2008). For example, the parent who lacks positive parenting skills may become more restrictive and negative when trying to cope with their non‐compliant child; this, in turn, makes the child more difficult to handle, which further increases parental distress and sense of helplessness and hopelessness in managing the child's misbehaviour (Campbell 1997).
Poor quality parenting is just one of a number of complex, interacting dispositional and contextual risk factors for conduct problems. Others include: individual differences amongst children; difficult temperament; impulsivity; low verbal intelligence; deficits in processing social information; neuro‐chemical abnormalities; economic deprivation; parental unemployment and low educational and occupational status; exposure to violence; deviant peer influences; and broader cultural factors (Patterson 1995; O' Connor 2002; Farrington 2002; Frick 2004). However, notwithstanding the effect of interacting and cumulative risk factors, there is growing evidence that an improvement in parenting skills can reduce problematic behaviours in childhood and increase children’s positive social and compliant behaviours, as well as improving parental mental health (Osofsky 2000; Patterson 2002; Webster‐Stratton 2004; Hutchings 2007). In particular, existing research supports the effectiveness of group‐based parenting interventions that are informed by behavioural, cognitive and social‐learning theory principles, in reducing the intensity of childhood conduct problems (Brestan 1998; Webster‐Stratton 2004; NICE 2006).
Description of the intervention
Behavioural and cognitive‐behavioural interventions incorporate social learning principles and techniques from cognitive therapy alongside principles of operant and classical learning. Such behavioural and cognitive‐behavioural parenting interventions are now typically delivered in a group format and have become increasingly popular as a means of addressing conduct problems in childhood (Sanders 2000; Sharry 2004; Webster‐Stratton 2004; Carr 2006).
Behavioural/cognitive‐behavioural group‐based parenting programmes typically involve an interactive, collaborative, learning format in which programme facilitators teach key behavioural principles and parenting skills (e.g. play, praise, rewards, discipline) to parents/caregivers who then practise the skills that they have learned. Key elements of effective programmes appear to include the following: learning how and when to use positive parenting skills; observation; modelling; behaviour rehearsal (e.g. role‐play); discussion; homework assignments; using peer support; and reframing unhelpful cognitive perceptions about their child or about child‐management in general (Gardner 2004; Hutchings 2004; Azar 2006). However, these behavioural/cognitive‐behavioural parenting programmes vary in the extent to which they include these components, and in the extent to which they appear to be effective with those families who are most at risk, or those children with the most severe problems (Hutchings 2004; Hutchings 2006).
How the intervention might work
Operant learning theory emphasises the environmental antecedents and consequences of behaviour. Thus, programmes based on operant learning theory teach techniques of positive and negative reinforcement to parents by helping them to focus on the child's positive behaviour (i.e. by praising and rewarding the desired behaviour) and to ignore or introduce limit‐setting and 'time‐out' consequences for the child's negative behaviour (Webster‐Stratton 2004). Parents also learn how to pinpoint proximal and distal antecedents for identified positive and negative target behaviours for their child (Webster‐Stratton 2000).
Social learning theory posits that children learn how to behave by imitating the behaviour modelled by others in their environment and so if this behaviour is reinforced, it is likely to be repeated (Bandura 1986). Such programmes focus on helping parents to model more effective behaviour. In addition, group facilitators/leaders have the opportunity, within certain group‐based behavioural programmes, to model key parenting skills in each session whilst parents imitate and practise the new skills through role‐plays and homework assignments (Webster‐Stratton 1998). Parents can also be encouraged to act as empathic and supportive role‐models for each other (Webster‐Stratton 1998). However, it is important to note that the level of role‐modelling and support provided by facilitators and other parents varies between programmes (Sanders 2000; Hutchings 2004).
The cognitive component focuses on problematic thinking patterns in parents, that have been associated with conduct problems in their children (Azar 2006). Typical cognitive distortions include, for example, globalised 'all or nothing' thinking. Thus, one minor setback may trigger a negative automatic thought (NAT) (e.g. 'I am a bad parent'), thereby leading to feelings of stress, hopelessness, low self‐esteem, a perceived inability to cope with the situation and learned helplessness (Seligman 1990). Therefore, behavioural/cognitive‐behavioural parenting interventions also seek to help parents learn how to reframe distorted cognitions or misattributions and to coach them in the use of problem‐solving and anger management techniques (Macdonald 2004).
Research suggests that the impact of behavioural/cognitive‐behavioural parenting interventions may be moderated by socio‐economic factors, such as socio‐economic disadvantage arising from low levels of income and/or low levels of educational attainment and employment (Hutchings 2004). For instance, Reyno 2006 found that socio‐economic status moderated the effectiveness of parenting programmes. Another important moderator of impact may include implementation fidelity, i.e. the extent to which programmes delivered in more naturalistic, service settings adhere to the original design of the programme. Thus, if monitoring (i.e. training and supervision of programme deliverers) is critical to programme success, a programme might be efficacious within experimental research settings but not effective when rolled out within more naturalistic settings (Mihalic 2002).
There is increasing evidence that an important mechanism of change within behavioural/cognitive‐behavioural group‐based interventions may involve change in parenting skill as a substantial predictor of child problem behaviour outcome (Gardner 2006). On the other hand, qualitative studies highlight the increased parental social support and confidence that comes from sharing problems within a group context (Barlow 2001; Patterson 2005). Therefore this review will explore putative mechanisms of change by examining change in (1) parenting skill and (2) parental social support and confidence as predictors of child behaviour outcomes.
Why it is important to do this review
Several studies have demonstrated the effectiveness of behavioural/cognitive‐behavioural group‐based parenting programmes in reducing conduct problems in children (e.g. Sanders 2000; Sharry 2004; Webster‐Stratton 2004). In addition, a number of previous reviews in the area have focused on the wide range of parenting programmes that are currently available and have produced evidence to suggest that group‐based interventions, based on social learning theory, offer an effective treatment for conduct problems in children (e.g. Brestan 1998; Barlow 2000; Farmer 2002; NICE 2006; Dretzke 2009). However, a number of key questions remain unanswered and this review of primary studies will address these.
Despite the number of reviews conducted in the area of parenting programmes, there has been growing concern that many of these do not use sufficiently rigorous methodological criteria in determining the quality of the evidence for included group‐based parenting interventions (Hutchings 2004; Dretzke 2009). For instance, in the NICE 2006 review, a parenting intervention based on only a single study, which had not achieved replication or long‐term follow‐up, nor demonstrated any tools for assessing implementation fidelity, received the same status as a programme that fulfilled all these criteria. Thus, the current review will employ sensitivity analyses to differentiate between studies with, and without, such an evidence base.
Hutchings 2007 and Dretzke 2009 also note that many previous reviews in the area have: included non‐randomised studies; failed to undertake an intention‐to‐treat analysis; did not test for heterogeneity prior to conducting the meta‐analysis; and failed to report confidence intervals. Dretzke 2009 addressed these issues in their review and aimed to employ Cochrane criteria in their meta‐analysis. However, their review was restricted to a limited number of child behaviour outcomes and did not examine the impact of the parenting programme on parental mental health. The current review will address these shortcomings. In addition, it will incorporate an evaluation of the cost‐effectiveness of parenting programmes, and investigate the moderators of impact, including socio‐economic status and implementation fidelity. Mediator analysis will also explore putative mechanisms of change by examining change in (i) parenting skill and (ii) parental social support and confidence as predictors of child problem behaviour outcomes.
Objectives
To examine the effectiveness of behavioural/cognitive‐behavioural group‐based parenting interventions for children with early onset conduct problems in improving a) child behaviour outcomes and b) parenting skills and parental mental health.
To critically appraise and summarise current evidence on the incremental resource use, costs and cost‐effectiveness of behavioural/cognitive‐behavioural group‐based parenting interventions when compared to treatment as usual.
Methods
Criteria for considering studies for this review
Types of studies
Randomised controlled trials (RCTs) with or without cluster randomisation, and quasi randomised studies (i.e. allocation by alternate days, date of birth), conducted in either research or service settings. The review will evaluate behavioural/cognitive‐behavioural group‐based parenting interventions compared with control conditions of waiting list, treatment as usual, or no treatment. Head‐to‐head studies comparing two different types of parenting programme without a control group will be excluded. Studies that report on conduct problems co‐morbid with ADD/ADHD will only be included if they report outcomes for conduct problems separately from ADD/ADHD outcomes.
For the economic evaluation, costs and cost‐effectiveness analyses will be included of behavioural/cognitive‐behavioural group‐based parenting interventions versus treatment as usual or no treatment. Only costs or cost‐effectiveness studies conducted alongside, or subsequent to RCTs that meet eligibility criteria, will be included (Shemilt 2008).
Types of participants
Parents, or primary caregivers of children aged three to 12 years who are manifesting either: (a) conduct problems, as identified by a score above the clinical cut‐off point on an outcome measure, such as the Eyberg Child Behaviour Inventory (ECBI) (Eyberg 1980); or (b) have a clinical/psychiatric diagnosis of Conduct Disorder (CD) and/or Oppositional Defiant Disorder (ODD), as classified by DSM‐IV 2000, or the International Statistical Classification of Diseases and Related Health Problems, 10th Revision (WHO 2009). Samples may be drawn from community or clinical settings. Primary caregivers may be of either gender and may be single parents or two‐parent families. Studies involving parents of children older than 12 or younger than three, will only be included where more than 90% of the sample fall within the age range specified above.
Types of interventions
Structured group‐based parenting programmes underpinned by behavioural/cognitive‐behavioural theories, and provided on a regular basis (e.g. weekly or fortnightly) for at least three sessions of between one and two hours.
Studies will be excluded in which parenting skills‐training forms a minor component of a larger programme. Education, or information‐based training programmes (i.e. that rely on providing information and/or discussion) will also be excluded.
Types of outcome measures
Primary outcomes
(1) Child Outcomes
A. Behavioural
i) Conduct problems
(2) Parenting Outcomes
A. Self‐report and direct observation of appropriate parenting skills and knowledge
B. Parental mental health
i) Stress, depression, anxiety levels, sense of confidence
Secondary outcomes
1. Child outcomes
A. Emotional
i) Depression/anxiety
B. Educational and cognitive ability
C. Long‐term outcomes in adolescence and adulthood
i) Criminal justice system involvement ‐ police contacts, court appearances, imprisonment
ii) Unemployment
2. Parenting outcomes
i) Increased level of social support
3. Adverse outcomes
A. Financial and psychological burden to family in attending and accessing course, e.g. child‐care issues
B. Increased conflict within family in relation to introduction of new parenting techniques
4. Economic data
The review of economic costs will be informed by guidelines outlined by Shemilt 2008. We have also enlisted the help of a health economist (CO'N) for this part of the review.
A. Costs per parent of running programme
i) Non‐recurrent costs: materials (programme kit), training of deliverers of programme
ii) Recurrent costs:
(1) Staff costs (salary per hour) in delivering programme, including delivering session, preparation, travel and supervision
(2) Facilities provided for parents (e.g. transport, creche, money for baby‐sitting, refreshments provided)
(3) Managerial overheads (e.g. venue rental)
B. Utilisation of health, social care and special education services by children and parents at different time‐points, e.g. six month follow up, 1 year follow up
i) Number and costs of visits to primary care (e.g. doctor, nurse, hospital, speech therapists, paediatricians)
ii) Social services (child psychology, psychiatric and social work services)
iii) Special education services (e.g. resource hours, special needs assistant)
C. Incremental Cost‐Effectiveness Ratios (ICER) at different follow‐up time points (i.e. an ICER point estimate compares the costs and consequences of running a behavioural/cognitive behavioural parenting intervention relative to the costs and consequences of a specified alternative (most commonly chosen to be the status quo)).
ICERs are a central component of full economic evaluations. However, full economic evaluations of parenting interventions may be relatively rare (Edwards 2007) and as such, we will also search for studies that include only costs data.
Data sources
Primary and secondary outcomes (one to three) may be measured (i) by parent, carer or child reports or (ii) by observational data that have been coded by one or more independent observer(s).
Timing of outcome assessment
All outcomes will be measured at baseline and at follow‐up at least three months following baseline. Follow‐ups of six months, twelve months and longer, post‐baseline will also be analysed, when such data are available.
Search methods for identification of studies
Studies will be identified through keyword searches of all relevant electronic databases, government policy databanks and professional websites, as well as searching grey literature (conference papers, unpublished PhDs), reference lists and personal communication with experts in the field.
Electronic searches
The following databases will be searched:
Cochrane Library, Cochrane Central Register of Controlled Trials (CENTRAL)
MEDLINE
Academic Search Premier (EBSCO Publishing)
ACM Digital Library
Annual Reviews
ASSIA (Applied Social Sciences Index and Abstracts)
CINAHL
Directory of Open Access Journals
Dissertations and Theses Abstracts
EMBASE
ERIC
IngentaConnect
JSTOR
metaRegister of Controlled Trials (mRCT)
Oxford Reference Online: Premium Collection
PsycArticles
PsycINFO
Psychological and Behavioural Sciences
Science Citation Index Expanded
Science Direct
Social Science Citation Index
Social Services Abstracts
Sociofile
Sociological Abstracts
Web of Knowledge
Wiley Interscience
Wilson Omniscience Full Text, Mega Edition
Economic sources:
NHS Economic Evaluation Database
European Network of Health Economic Evaluation Databases
Health Economic Evaluations Database
CEA Registry
Econlit
Paediatric Economic Database Evalation
Government policy sources:
Acompline
Center for the Study and Prevention of Violence, U. S.
HSE Libraries Online (Ireland)
National Institute for Health and Clinical Excellence (NICE)
U.S. Department of Health and Human Services
U.S. National Institutes of Health, CRISP database
U.S. Government Printing Office
UK Home Office
Search terms
Search terms for MEDLINE, (modified as necessary for other databases),can be found in Appendix 1. There will be no date, publication, geographic, or language restrictions applied to the searches.
Searching other resources
Other sources of information will also be searched. Reference lists of pertinent articles and of systematic and non‐systematic reviews identified through database searches, will be examined to identify further relevant studies. In addition, experts and researchers working in the area, will be contacted in order to search for unpublished and ongoing studies (e.g. conference papers, unpublished dissertations, working papers or government reports).
Data collection and analysis
Selection of studies
Titles and abstracts identified through searches will be reviewed independently by MF and TB in order to determine their potential eligibility against the above inclusion criteria. Any citation deemed potentially relevant by at least one author will be retrieved in full text and again, independently assessed against the inclusion criteria by MF and TB. Disagreements will be resolved by consensus with a third author (SMcG in consultation, where necessary, with JH, MD or SS). Study authors will be contacted to obtain additional information, if necessary. Specific reasons for exclusion will be documented for each study that might reasonably have expected to have been included, but which did not meet the inclusion criteria. Studies in other languages for which we cannot obtain translators will be referred to the editorial base of the CDPLPG.
Data extraction and management
Information on study design and implementation, sample characteristics, intervention and control characteristics, implementation integrity, and on all reported outcomes will be independently extracted by MF and TB, using a data extraction form developed for this review. This will be drafted by two authors, piloted on a sample of studies, and then finalised. Completed data extraction forms will be kept with each paper. If differences occur between coders in extracting data from studies, these will be discussed in order to resolve discrepancies and to refine the data extraction form if necessary. If agreement cannot be reached, then SMcG will arbitrate, in consultation, where necessary, with JH, MD or SS. If missing data means that disagreement cannot be resolved, then additional information will be sought from the authors of the study. Citations and data will be entered and organized in RevMan 5.
For the economic analysis, details of the characteristics and results of included health economics studies will be collected. Characteristics may include: year of study; details of interventions and comparators; study design; source(s) of resource use; unit costs; decision making jurisdiction; geographical and organisational setting; analytic perspective; discount rates and time horizon for both costs and effects. Results will include the type and quantity of each resource used and its unit cost as associated with interventions and comparators; price year and currency will be used to calculate costs and incremental costs (Shemilt 2008).
Assessment of risk of bias in included studies
Review authors will independently assess the risk of bias within each included study in relation to the following six domains on the basis of ratings of 'Yes' (low risk of bias); 'No' (high risk of bias) and 'Unclear' (uncertain risk of bias) (Higgins 2008): sequence generation; allocation concealment; blinding; incomplete outcome data (including data on attrition and exclusions; differentiating intention‐to‐treat analyses from per‐protocol [“as treated”] analyses); selective outcome reporting; implementation integrity and other sources of bias. These judgements will be entered into a 'Risk of Bias' table in RevMan 5 with a brief rationale for the judgement. SMcG will arbitrate in the event of any disagreement between MF and TB. Essential information concerning ‘risk of bias’ criteria will be sought from study authors where necessary. If such essential information is not available from study authors, the review team will then consult with the CDPLPG editorial base. In addition, studies in other languages will be referred to the CDPLPG editorial base for assistance with translation and data extraction.
For the cost‐effectiveness analysis, the 'Drummond checklist' and 'Evers checklist' will be used, in conjunction with the NHS EED structured abstract, where available, to critically appraise the methodological quality of included health economic studies (Shemilt 2008).
Measures of treatment effect
Continous data
Continuous data will be analysed if means and standard deviations are available, or if there is some other way to calculate effect size (e.g. from t‐tests, F‐tests, or exact p‐values). If reports have insufficient data, additional information will be requested from the authors. Where scales measure the same clinical outcomes (e.g. child’s conduct problems), in different ways, standardised mean differences (SMD) or Cohen’s d (Cohen 1969), will be estimated using RevMan's formula for SMD; this is based on Hedges’ g which includes an adjustment for small sample bias. Inverse variance methods will be used to pool SMDs. A weighted mean difference will be used where studies employ the same measures. Confidence intervals of 95% will be used for individual study data and pooled estimates throughout.
Binary / dichotomous data
For binary/dichotomous outcomes, we will calculate odds ratios and their 95% confidence intervals. RevMan 5 uses Mantel‐Haenszel methods for combining binary outcome data across studies. When some primary studies report an outcome as a dichotomous measure and others use a continuous measure of the same construct, we will first conduct two separate meta‐analyses (one for odds ratios and another for SMDs). Next, in order to increase the statistical power of the meta‐analyses, odds ratios will be converted to d indices using the Cox formula (log odds ratio divided by 1.65 (Sanchez‐Meca 2003)) and we will perform another meta‐analysis that includes all possible studies. For studies that provide both dichotomous and continuous measures of the same construct, study average effect sizes (ES) will be calculated with Hedges' g.
Due to expected heterogeneity, a random‐effects meta‐analysis will be employed. The random‐effects model generally provides a more conservative result and takes into account the fact that various studies are estimating different, yet related, intervention effects (Deeks 2001).
If studies report multiple measures of the same construct at different points in time, we will conduct separate meta‐analyses for outcomes measured at several periods of follow‐up: at three months, six months, one year and two years post‐baseline (and longer follow‐up points, if such data are available).
Economic evaluation
Studies will initially be classified according to whether they measure resource costs or whether they calculate an incremental cost‐effectiveness ratio (ICER). An ICER point estimate compares the costs and consequences of running a behavioural/cognitive behavioural parenting intervention relative to the costs and consequences of a specified alternative (most commonly chosen to be the status quo). Studies will be assessed for risk of bias, using the Drummond and Evers checklists, before a decision is made to pool any studies, particularly in relation to whether the metric in question has equivalent meaning across studies (Shemilt 2008). In circumstances where there is evidence of little variation in resource or cost use between studies, it may be regarded as legitimate to present a pooled estimate. Otherwise we will clearly present the distribution of costs (Shemilt 2008).
If a decision is made to conduct meta‐analyses of resource use or cost data, this will be supported by a thorough critical appraisal of the methods used to derive such estimates within the corresponding health economics studies, alongside use of 95% confidence intervals and statistical methods to investigate and incorporate between‐study heterogeneity (e.g. I2 statistic, Chi2 test, random‐effects models). Cost estimates collected from multiple studies will be adjusted to a common currency and price year before these data are pooled. Careful consideration will be given to the jurisdiction, analytic perspective and time horizon for both costs and effects.
If meta‐analyses of resource use or cost data are conducted, a narrative summary will be included in the Results section to comment on the direction and magnitude of results and their precision. Similarly, if two or more health economics studies are included in a review, but a decision is taken not to pool (in a meta‐analysis) resource use and/or cost data that have been collected from these studies, this will be stated in the Methods section (Shemilt 2008).
Any economic analysis undertaken in this review will be either conducted by or overseen by our health economist.
Unit of analysis issues
The authors will take into account the level at which randomisation occurred to determine whether individuals were randomised in groups (i.e. cluster randomised trials), whether results were reported at multiple time points, or whether individuals may have undergone multiple interventions simultaneously.
Cluster randomised trials
Data will be controlled for clustering, using the procedures outlined by Higgins 2008. When outcome measures are dichotomous, the number of events and number of participants per trial arm will be divided by the design effect [1 = (1‐m) * r], where m is the average cluster size and r is the intra‐cluster correlation coefficient (ICC). When outcome measures are continuous, the number of participants per trial arm will be divided by the design effect, with the mean values unchanged. To determine the ICC, authors will use estimates in the primary trials on a study‐by‐study basis. If these values are not reported, the authors will use external estimates of the ICC that are appropriate to each trial context and average cluster size (Higgins 2008).
Multiple time points
Each outcome at each time‐point will be analysed in a separate meta‐analysis, as outlined above.
Dealing with missing data
Missing data and drop outs/attrition will be assessed for each study. Numbers, reasons and characteristics of drop outs will be assessed and reported. Authors will be contacted for missing data. Any meta‐analysis will use data from all original participants where possible and will report when that is not the case. If missing data are not available in studies, a sensitivity analysis will be conducted to assess potential bias in the analysis. The extent to which the results might be biased by missing data will also be discussed.
Assessment of heterogeneity
Heterogeneity among included studies will be assessed visually and by examining I2 (Higgins 2002), a quantity which describes the approximate proportion of variation in point estimates that is due to heterogeneity rather than sampling error. This will be supplemented by the Chi2 test, where a low p‐value indicates heterogeneity of treatment effects and by comparing the results of fixed‐effect and random‐effects models (Deeks 2008). If there is evidence of heterogeneity (e.g. statistically significant Chi2 tests and I2 greater than 50%), authors will discuss possible reasons for it and conduct sensitivity analyses accordingly; the issue of study size and power will be considered in the interpretation of the results. Subgroup analyses may be used to investigate this further.
Studies will be considered appropriate for meta‐analysis when they show comparability/lack of heterogeneity in terms of participants, interventions, controls and outcomes. Studies will not be included within a meta‐analysis if, for example, they show risk of bias or serious publication bias or if outcomes are too diverse. For instance, child outcomes measuring externalising conduct problems and internalising depressive and anxiety symptoms will not be combined. Neither will parent and child outcomes be combined. In addition, separate meta‐analyses will be conducted for outcomes at different time‐points. If suitable numerical data are not available for meta‐analysis, or if meta‐analyses are considered inappropriate, an organised narrative account will be provided.
For the economic evaluations, careful attention will be given to whether the metric in question has equivalent meaning across studies before data are pooled (Shemilt 2008). Cost estimates collected from multiple studies will be adjusted to a common currency using purchasing power parity and price year before these data are pooled. Between‐study heterogeneity will be assessed using the Chi2 test, the I2 statistic and random‐effects models.
Assessment of reporting biases
Funnel plots will be drawn, where possible, to investigate any relationship between effect size and standard error. Such a relationship could be due to publication or related biases, or due to systematic differences between small and large studies. Where such a relationship is identified, the experimental diversity of the studies will be further examined as a possible explanation (Egger 1997).
Data synthesis
Data synthesis will be conducted with RevMan 5, the latest version of the Cochrane Collaboration's meta‐analysis software. When meta‐analysis is inappropriate, a narrative description of the study results alone will be provided.
Subgroup analysis and investigation of heterogeneity
Further investigations of the causes of heterogeneity may be conducted using subgroup analysis. Possible subgroups that may be examined, if present, will include:
-
Children with more severe behaviour problems pre‐treatment (i.e. conduct problems versus children with a diagnosis of CD)
-
Socio‐economic disadvantage (e.g. low income, low education/occupation)
-
Research versus service settings
-
Implementation fidelity of programme (i.e. training and supervision of facilitators delivering the programme)
Meta‐regression: mediator/mechanisms of change analysis
It is important to examine the mechanisms of change within an intervention. We will explore whether change in parenting skill and increased parental social support/derived confidence act as predictors of child behaviour outcome (Barlow 2001; Patterson 2005; Gardner 2006). The regression coefficient obtained from a meta‐regression analysis will describe how the outcome variable (the intervention effect) changes with a unit increase in the explanatory variable (i.e. positive parenting skill or increased parental support). The statistical significance of the regression coefficient is a test of whether there is a linear relationship between intervention effect and the explanatory variable. We will use a random‐effects meta‐regression so as to allow for the residual heterogeneity among intervention effects not modelled by the explanatory variables. Meta‐regression will not be conducted when there are fewer than ten studies in a meta‐analysis (Deeks 2008). Meta‐regression will be performed using the 'metareg' macro available for the Stata statistical package.
We are aware that the likelihood of a false positive result among subgroup analyses and meta‐regressions increases according to the number of characteristics investigated. Thus, it may be necessary to adjust the level of significance in order to take the multiple comparisons into account (Yusuf 1991). However, it is important to examine these in view of research to suggest that these subgroup factors might impact upon programme outcomes (Thompson 2002).
Sensitivity analysis
Sensitivity analyses will be performed to evaluate the robustness of the pooled effect sizes across various components of methodological quality, including:
-
Randomisation versus quasi‐randomisation
-
Removal of studies which have not achieved independent replication, and removal of studies that do not report on quality assurance/fidelity
-
Short term versus long‐term follow‐up of primary outcomes
-
Exclusion of studies with attrition rates larger than 20%
-
Changing the way that values are imputed for missing data (e.g. last observation carried forward versus mean scores for missing values)
-
Standardised versus non‐standardised measures of outcomes (standardised scales are those that are validated in a peer‐reviewed journal or validated against other similar measures)
-
Studies with blind assessment of outcomes versus those without blind assessment of outcomes
