Child‐focused psychosocial interventions for anger and aggression in children under 12 years of age
Abstract
This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:
To evaluate the effectiveness of child‐focused psychosocial interventions for anger and aggression in children under 12 years of age.
Background
Description of the condition
Coping with anger in childhood can be a challenge for children who do not have the skills to manage their feelings. Inability to cope with anger can lead to aggression and other externalising problems (Garner 2010). Aggression has been defined as "Any form of behaviour directed toward the goal of harming or injuring another living being who is motivated to avoid such treatment" (Baron 1994, p 7). Aggression can be direct, such as physical violence, abusive or threatening behaviour, or indirect, such as spreading rumours or deliberate social exclusion. When children have the awareness and skills to cognitively process negative feelings, such as anger, they can choose more adaptive action and avoid resorting to aggressive or destructive behaviour (Feindler 1984; Lochman 1981).
While it is difficult to find direct measures of the prevalence of anger and aggression problems in children and young people, indirect measures indicate that anger and aggression are significant social problems. Prevalence estimates of externalising behaviour problems, which includes aggression but also hyperactivity and delinquency, are estimated to be between 13% and 14% of the population under 18 years of age in Australia (Sawyer 2000). Surveys of adolescent mental health in the UK give prevalence estimates of between 6% and 16.7% for "conduct problems" in young people (Collinshaw 2004). Boys have higher prevalence rates than girls (Collinshaw 2004; Collinshaw 2010), but this gap closes with age (see Connor 2002 for review). In Norway, 5% of 12‐ to 15‐year olds self reported behaving aggressively towards others 'often', with 10% reporting being victims of other children's aggressive behaviour (Undheim 2010). In Northern Ireland, 'violence against the person' accounted for 70.6% (in 2011/2012) and 68.7% (in 2012/2013) of crimes committed by people under 18 years of age (PSNI 2013). More specifically worldwide prevalence estimates of conduct disorder in 5‐ to 19‐year olds range from 1.5% in girls to 3.6% in boys (Erskine 2013). Problems with anger may be increasing; Collinshaw 2010, in a study of mood disorder symptoms, identified a large and significant increase in reports of 'irritability' in boys and girls in England from 1986 to 2003.
Aggression in children tends to peak in early childhood between two and four years of age (Piquero 2012; Tremblay 2010), with the majority of children learning socially acceptable ways to deal with their environment as they grow up. However, some children (estimated at 15%, Piquero 2012) fail to reduce their aggressive behaviour and exhibit a stable trajectory of aggression and anti‐social behaviour into adolescence and beyond. Problems with managing anger and aggression can impact negatively on children in a variety of ways, including school exclusions, social problems, externalising behaviour problems, internalising behaviour problems, poor emotional health and well‐being, and involvement in the criminal justice system. Early persistent aggressive behaviour is linked to adult criminality (Piquero 2012). Prospective longitudinal studies have linked childhood aggression to domestic violence (Temcheff 2008), and poorer health outcomes in adulthood for both men and women (Temcheff 2011). Without appropriate intervention, young children with anger and aggression problems may develop serious, chronic anti‐social behaviour. Given the prevalence and long‐term impact of anger and aggression problems in childhood, it is important that effective interventions are identified and implemented and ineffective interventions are discontinued.
Description of the intervention
Some interventions target anger through modifying the environment or other people's behaviour. Such interventions will not be the focus of this review; there are already a number of Cochrane systematic reviews that focus on the family as means of addressing children's behaviour (Furlong 2012; Littell 2005; Turner 2007; Woolfenden 2001). This review will focus on psychosocial interventions designed to enable children to manage their own behaviour more effectively. Intervention targets range from universal (e.g. school‐based interventions for all students), to selective prevention (targeting children at risk or showing early signs of disorder), and indicated treatment (for children with clinical diagnosis of a conduct disorder).
Psychosocial interventions use psychological or social strategies, or both, to improve children's ability to manage anger effectively and consequently reduce aggressive behaviour. The term 'anger management' is commonly used to describe these types of interventions. While individual interventions vary in their theoretical approach and the components they include, they typically use one or more of the following strategies: relaxation techniques, social skills and coping skills training (including role play, modelling appropriate behaviour and mindfulness techniques), emotion or self regulation skills training, developing adaptive information processing and social problem solving.
Anger management interventions typically use a cognitive behavioural approach although other approaches, such as mind‐body interventions, have also been investigated. Cognitive‐behavioural therapies target behaviour and patterns of thinking in order to address a person's difficulties. Examples include anger control training, the Coping Power Program and social skills training.
-
Anger control training (Feindler 1984; Feindler 1986) is based on Novaco's anger control intervention for adults (Novaco 1975) and Meichenbaum's stress inoculation model (Meichenbaum 1973). It comprises three modules that focus on arousal management (through relaxation techniques), social‐problem solving (by targeting the social‐cognitive mediators of anger), and social skills training (modelling and practicing adaptive social behaviour).
-
The Coping Power Program (e.g. Lochman 2003) is based on anger control training, but has a greater emphasis on social problem solving and less focus on managing arousal.
-
Social skills training (e.g. Sukhodolsky 2006) emphasises the importance of understanding appropriate social behaviour and helping children to acquire key social skills, such as communication and conflict management, so that they can choose and implement adaptive behaviour in place of anger and aggression.
-
Mind‐body approaches, such as mindfulness meditation, yoga, and biofeedback training, are examples of therapeutic approaches that typically focus on reducing arousal through relaxation and greater awareness of bodily sensations. Continued practice is thought to increase self regulation skill (e.g. Khalsa 2012). More recently, mindfulness techniques have been integrated with more typical cognitive‐behavioural approaches, for example, in mindfulness‐based stress reduction (MBSR).
How the intervention might work
There are a number of mechanisms through which these interventions might work and many interventions target more than one factor that is thought to contribute to children's anger problems.
Cognitive distortions or biases towards processing neutral information as threatening can lead to increased anger and aggression. Correcting maladaptive, distorted social‐information processing should lead to decreased feelings of anger and lower likelihood of reacting aggressively. This is usually a core component of cognitive approaches that is generally absent in psychoeducational or skills training approaches.
Cognitive deficits in social problem solving can lead to selection of maladaptive aggression to solve social problems. Children who display frequent anger and aggression do not have the social skills necessary to navigate their social world effectively. Equipping children with more adaptive ways to solve social problems can reduce aggression (Dodge 1986).
Arousal management refers to a person's ability to control their emotional arousal. Children who lack awareness of their emotional state or who do not recognise triggers for anger and aggression will have great difficulty in managing their emotions. Arousal management involves becoming aware of emotional states and learning and practicing skills to control them. The goal is to improve both recognition of angry feelings and ability to address them before they escalate into aggression. Techniques include relaxation training, yoga and meditation.
Why it is important to do this review
Reducing anger and aggression in children and young people has the potential to reduce offending behaviour and school exclusions, and improve children's overall emotional health and well‐being. However, the range of interventions is large. Parent‐focused interventions have been comprehensively reviewed and are now currently recommended by NICE (National Institute for Health and Care Excellence) guidelines for anti‐social behaviour and conduct disorder for children between seven and 17 years of age (NICE 2013). To date, four separate Cochrane reviews have evaluated the effectiveness of family, parenting or multi‐systemic interventions for conduct problems and delinquency in children aged three to 12 years (Furlong 2012), children in adolescence (Littell 2005; Woolfenden 2001), and children in foster care (Turner 2007). Parent‐focused interventions have also been evaluated in other comprehensive systematic reviews for disruptive behaviour (Michelson 2013), conduct disorder (Bonin 2011; Dretzke 2005), and oppositional defiant disorder (Bradley 2005).
Child‐focused interventions that specifically address anger and aggression problems, including children with conduct disorder, have not yet been the subject of a Cochrane review. Existing meta‐analyses indicate that psychosocial anger management interventions may be effective in children and adolescents for reducing anger and aggression in both clinical (Fossum 2008) and non‐clinical populations (Sukhodolsky 2004; Wilson 2007). However, the majority of existing reviews focus only on school‐based interventions (Barnes 2014; Durlak 2011; Gansle 2005; Hahn 2007; Kuhn 2015; Mytton 2006; Wilson 2007) at the exclusion of community, forensic and psychiatric settings, where most need is likely to be found. Other relevant reviews limited their inclusion criteria to English language studies only (Candelaria 2012), or children and adolescents with severe problems (Fossum 2008; Hoogsteder 2015), or focused solely on adolescents (Hoogsteder 2015; Kuhn 2015).
A number of related reviews focus on violence prevention interventions (Hahn 2007; Limbos 2007; Matjasko 2012; Mytton 2006; Özabacı 2011). Most relevant is the Mytton 2006 Cochrane review, which assessed school‐based programmes for preventing violence and indicated that long‐term benefits in primary school age children were uncertain. Two other reviews also suggested that interventions appeared to be more effective in older children (Bennett 2000; Sukhodolsky 2004). The effectiveness of interventions in younger children is not yet established and there are currently no reviews that focus specifically on younger children across any setting (school, forensic, psychiatric, community).
Objectives
To evaluate the effectiveness of child‐focused psychosocial interventions for anger and aggression in children under 12 years of age.
Methods
Criteria for considering studies for this review
Types of studies
Randomised controlled trials (RCTs) and quasi‐randomised controlled trials (qRCTs) (in which allocation is done on the basis of a pseudo‐random sequence such as alternation or hospital number).
Types of participants
Children between birth and 12 years of age.
If we identify studies that include children in an overlapping age range, we will contact the authors to access individual participant data for children that meet our criteria. If individual participant data are unavailable, we will only include studies with a mean age of less than 12 years.
We will exclude studies in which the population is children with a developmental disorder, including autism spectrum disorder and intellectual disabilities.
Types of interventions
Any psychosocial intervention aimed at helping children to improve their anger control and reduce their aggressive behaviour.
We will exclude interventions that modify only the environment, parent‐focused interventions and pharmacological treatment.
Relevant comparisons may include no intervention, wait‐list controls, treatment‐as‐usual or a comparison intervention.
Types of outcome measures
Where feasible, we will make comparisons at the following time points: up to one month post intervention, six months' follow‐up, 12 months' follow‐up, two years post intervention, and any period of time more than two years post intervention.
Primary outcomes
-
Anger (e.g. State Trait Anger Expression Inventory 2 child and adolescent (STAXI‐2 C/A), self, parent or teacher report)*.
-
Aggressive behaviour, including threatening and violent behaviour and acts of aggression (e.g. Measure of Aggression, Violence, and Rage (MAVRIC) or self, parent, teacher or peer report)*.
-
Adverse effects*: stigmatisation (e.g. a modified, child appropriate version of the Discrimination and Stigma Scale (DISC), Versions 10 (DISC‐10) or 12 (DISC‐12), by Brohan 2013).
Secondary outcomes
-
Externalising and internalising behaviour problems (e.g. Child Behaviour Checklist; CBCL)*. If data are presented separately for an aggression subscale that forms part of an overall measure of externalising problems, we will include this as a primary outcome.
-
Self control (e.g. Ages and Stages Questionnaire: Social Emotional (ASQ: SE))*.
-
Social skills (e.g. ASQ: SE or Strengths and Difficulties Questionnaire (SDQ)).
*We will include these outcomes in a 'Summary of findings' table.
Search methods for identification of studies
Electronic searches
We will identify trials by searching all available years of the following databases.
-
Cochrane Central Register of Controlled Trials (CENTRAL), part of the Cochrane Library (which includes the Cochrane Developmental Psychosocial and Learning Problems Group Specialized Register).
-
Ovid MEDLINE.
-
Ovid MEDLINE(R) In‐Process & Other Non‐Indexed Citations.
-
Embase (Ovid).
-
PsycINFO (Ovid).
-
CINAHL Plus (EBSCOhost).
-
Science Citation Index (SCI) (Web of Science).
-
Social Sciences Citation Index (SSCI) (Web of Science).
-
Conference Proceedings Citation Index ‐ Science (CPCI‐S) (Web of Science).
-
Conference proceedings Citation index ‐ Social Science and Humanities (CPCI‐SS&H) (Web of Science).
-
ERIC (EBSCOhost).
-
British Education Index (EBSCOhost).
-
Cochrane Database of Systematic Reviews (CDSR), part of the Cochrane Library.
-
Database of Abstracts of Reviews of Effects (DARE), part of the Cochrane Library.
-
ProQuest Dissertations & Theses; UK & Ireland (for dissertations).
-
WorldCAT (limited to theses) (worldcat.org/).
-
The Campbell Library (campbellcollaboration.org/lib/).
-
World Health Organization (WHO) International Clinical Trials Registry Platform (apps.who.int/trialsearch).
-
ClinicalTrials.gov (clinicaltrials.gov).
-
National Registry of Evidence‐based Programs and Practices (NREPP) (nrepp.samhsa.gov/ViewIntervention.aspx?id=211).
We will use the search strategy in Appendix 1 to search Ovid MEDLINE and will adapt it for all other databases. We will apply no restrictions to date, language or publication status. We will apply search filters for RCTs where appropriate.
Searching other resources
In addition, we will search the reference lists of all included studies and existing reviews to identify other relevant studies. We will also contact authors of included studies and experts in the field to obtain additional data not presented in published studies and enquire about related unpublished trials.
Data collection and analysis
Selection of studies
We will store and sift our search results using EPPI Reviewer 4, review management software (EPPI‐Reviewer 4.0). One reviewer (JH) will remove duplicate records and obviously irrelevant records based on a preliminary screen of titles. Thereafter, two review authors (JH and NL or GM) will screen remaining titles and abstracts for eligibility, and will retrieve full‐text articles of potentially relevant studies. Then, two review authors (JH and NL or GM) will independently assess full‐text articles for inclusion against the selection criteria. At this stage, we will record and present reasons for study exclusion in the 'Characteristics of excluded studies' table. We will resolve any disagreements by discussion with all authors.
Data extraction and management
Two review authors (JH and GM or NL) will independently extract and store data in EPPI Reviewer 4 software (EPPI‐Reviewer 4.0). We will discuss disagreements until we reach consensus.
We will extract the following data:
-
general information: author(s), type of source (journal, conference proceeding, book, report, thesis, other), name of source, year of study, unique report ID, study ID (if multiple reports of same study are included), author contact details;
-
trial details: study design, randomisation, study location, duration, attrition;
-
participants' information: age, gender, ethnicity, recruited from or reasons for referral to the study, care status (living with parents, foster parents, residential care, etc.);
-
intervention and control characteristics: components of the intervention, setting (e.g. one‐to‐one, group, classroom), location (e.g. home, clinic, school), method of delivery, details of training received by intervention providers, number of sessions, duration of sessions, manualised delivery or not, target of the intervention (universal, indicated prevention, treatment of clinical level problems);
-
outcomes: any measures related to primary or secondary outcomes (see Types of outcome measures), outcomes measured versus outcomes reported, timing of data collection or follow‐up, measurement tools used, study drop‐out or completion rates. We will extract outcome data in all forms in which they are given (e.g. change data, endpoint data, data for each category on ordinal scales).
Assessment of risk of bias in included studies
Two review authors (JH and NL) will independently examine the risk of bias of the included studies. They will perform the 'Risk of bias' assessments in line with recommendations of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). This will be done using the Cochrane two‐part tool. First, we will extract descriptions of what happened in the study, and second, we will give a rating of the likely risk of bias for the adequacy of the following six domains.
-
Sequence generation (i.e. was the allocation sequence randomly generated using, for example, a random number generator or coin‐toss?).
-
Allocation concealment (i.e. was allocation concealed from researchers and participants until after decisions about eligibility were made?).
-
Blinding of participants, personnel and outcome assessors (i.e. was knowledge of the allocated intervention prevented during the study?).
-
Incomplete outcome data (i.e. was it clear how many people were randomised to each group and were any drop‐outs from the trial accounted for or was intention‐to‐treat analysis used, or both?).
-
Selective outcome reporting (i.e. were reports of the study free of suggestion of selective outcome reporting?). We will assess this by checking the trial protocol if available from trial registry or from study authors).
-
Other sources of bias (i.e. was the study apparently free of other problems that could put it at a high risk of bias?).
We will assess and assign each domain, for each included paper, to one of the following categories as described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).
-
High risk of bias (if answers to the above questions are no).
-
Low risk of bias (if answers to the above questions are yes).
-
Unclear or unknown risk of bias (if insufficient information is available to answer the above questions).
We will resolve disagreements between raters by discussion until we reach consensus.
Measures of treatment effect
We will identify skewed data using the technique outlined in Chapter 9 of the Cochrane Handbook for Systematic Reviews of Intervention (maximum scale score minus the mean divided by the standard deviation (Higgins 2011). A ratio of less than two indicates skew, while a ratio less than one indicates substantial skew). We will seek advice from a statistician if we identify substantial skew.
Dichotomous data
We will analyse dichotomous outcome data by calculating the risk ratio (RR) and its 95% confidence interval (CI).
Continuous data
We will analyse continuous outcome data by calculating the mean difference (MD) when outcomes are measured on the same scale or the standardised mean difference (SMD) when the same outcome is measured in different ways, with 95% CI.
Ordinal data
We will inspect ordinal data to ensure it is appropriate to pool data. To do this, two review authors (JH and NL or GM) will inspect the measures used and only pool data from instruments measuring the same underlying concept. We will assess this on the basis of studies that compare the measurement properties of each instrument. If there are no such studies the review authors will inspect and discuss each measure until a consensus is reached on which measures can be pooled in a meta‐analysis. If we cannot pool findings, we will present them narratively. If it is appropriate to pool data, we will calculate MD or SMD in the experimental and control groups. If it is not appropriate to use mean score difference, for example, where an ordinal scale is more usefully dichotomised for analysis, then we will treat data in the same way as dichotomous outcome data.
We will extract both change data and endpoint data. For analysis, we will prefer change data over endpoint data, if available. We will combine studies with change data in a meta‐analysis with studies with endpoint data using the (unstandardised) mean difference method in Review Manager 5 (RevMan 2014). We will present MD change data in one subgroup, MD in endpoint data in another, and pool both subgroups for an overall analysis (Higgins 2011).
Multiple outcomes
Where studies use multiple, interchangeable measures of the same construct at the same point in time (e.g. multiple measures of aggression), we will calculate the mean SMD across these outcomes, together with the mean of their estimated variances. This will avoid the need to select a single measure and consequent inflated precision in meta‐analyses (studies that report on more outcome measures will not receive more weight compared with those that report using only one measure).
Two review authors (JH and NL or GM) will look for studies that compare the measurement properties of measures to be combined and, if none are available, the review authors will inspect and discuss each measure until a consensus is reached on which measures can and cannot be combined.
If outcome data are reported at different time points then we will perform a separate analysis on post‐intervention outcome data according to the following time frames: up to one month (post intervention), one to three months (short term), six to 12 months (medium term) and 12 months or more post intervention (long term).
Review authors will document all decisions made regarding these issues in the review.
Unit of analysis issues
Cluster‐randomised controlled trials
It is likely that a number of interventions will be delivered in school settings where randomisation occurs at a class or school level (i.e. a cluster). If we identify cluster‐randomised controlled trials, we will adhere to the guidance on statistical methods for managing data from cluster‐randomised controlled trials provided by the Cochrane Handbook of Systematic Reviews of Interventions (Higgins 2011, Section 16.3). We will check that adequate adjustments for clustering were made for estimates of treatment effects. If not, we will seek to extract or calculate effect estimates and their standard errors as for a parallel group trial, and adjust the standard errors to account for the clustering (Donner 1980). This requires information on an appropriate intraclass correlation coefficient (ICC), which provides information on the relative variability in outcome within and between clusters (Donner 1980). If this information is not available in the relevant report, we will request the information from the study authors. If this is not available or we receive no response, we will use external estimates obtained from studies that provide the best match on outcome measures and types of clusters from existing databases of ICCs (Ukoumunne 1999) or other studies within the review. If we are unable to identify an appropriate ICC, we will perform sensitivity analyses using a high ICC of 0.1, a moderate ICC of 0.01 and a small ICC of 0.00. These values are rather arbitrary but, as it is unlikely that the ICC is actually 0, it is preferable to use them to adjust the effect estimates and their standards errors. We will combine the estimates and corrected standard errors from cluster‐randomised controlled trials with those from parallel designs using the inverse variance method in Review Manager 5 (RevMan 2014).
Multiple treatment groups
It is possible that relevant studies may make use of multiple intervention groups. In our primary analyses, we will combine data from all eligible intervention arms and compare these with data combined with all eligible control arms, making a single pair‐wise comparison. If it is not appropriate to combine interventions arms, for example, the study compares two distinct types of intervention (e.g. yoga and a coping power intervention), then we will analyse each intervention separately (against a common control group) but divide the samples size for the common comparison groups to avoid inappropriate double counting of individuals. For dichotomous outcomes, we will sum the sample sizes and number of events or people experiencing an event across groups. In the case of continuous outcomes, we will combine means and standard deviations using techniques described in Chapter 7 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).
Cross‐over trials
Should we identify any relevant studies in which participants received both the control and intervention treatment but in a different order, we will only use the data collected up to the first cross‐over point of the study, in order to avoid any potential problems such as a carry‐over effect. In the event that data were not reported separately for intervention effects up to the cross‐over point, we will contact authors to request the data. If we are unable to obtain these data from authors, we will not include the data from the relevant study in the meta‐analysis and will describe it narratively.
Dealing with missing data
Where necessary, we will contact authors of included studies to supply any unreported data such as group means and standard deviations and details of drop‐outs or interventions received by the control group.
Substantial levels of loss of follow‐up may call the validity of the results into question. As it is not yet universally agreed what degree of attrition leads to a high degree of bias, we have chosen to include data from all relevant trials, regardless of the percentage of participants completing them. We will describe the potential bias resulting from the inclusion of such studies using the previously described 'Risk of bias' tool.
For both dichotomous and continuous data, in the event that the authors applied an intention‐to‐treat analysis, we will use the results provided.
If authors did not apply an intention‐to‐treat analysis, and data are missing or not reported at the study level, we will first contact authors to request the missing values. If data remain unavailable after reasonable attempts to contact authors, and it is reasonable to assume that the data are 'missing at random', we will include the study data and analyse the data using an available case analysis.
If dichotomous data are not 'missing at random' (e.g. if it is likely the participant failed to complete the follow‐up assessments as they experienced a negative outcome or adverse event), we will impute the missing data with the assumption that the missing data are negative. We will explore the impact of this decision using sensitivity analysis.
If a relevant study does not provide the summary data needed for meta‐analysis (e.g. standard deviations), we will derive these where possible using calculations provided in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).
We will assess the sensitivity of any primary meta‐analyses to missing data using the strategy recommended by Higgins (Higgins 2008).
Assessment of heterogeneity
We will first assess clinical heterogeneity by considering the distribution of key participant factors (age, gender, inclusion criteria) and trial factors (intervention type as described by study authors, randomisation, concealment, blinding of outcome assessment, losses to follow‐up). Should we identify any unexpected variability in these areas, we will discuss it in full. We will describe statistical heterogeneity by computing the I² statistic (Higgins 2002), a quantity that describes approximately the proportion of variation in point estimates that is due to heterogeneity rather than sampling error. In addition, we will conduct the Chi² test of homogeneity to determine the strength of evidence regarding the genuineness of that heterogeneity. If heterogeneity is substantial (defined as an I² of at least 50%, and a statistically significant Chi² statistic (P value < 0.10)), we will explore reasons for heterogeneity (see Subgroup analysis and investigation of heterogeneity).
Assessment of reporting biases
We will minimise reporting bias by comprehensive searching, contacting authors in the field and not limiting searches by language or publication type. Where there are a sufficient number of studies, usually considered to be a minimum of 10, we will assess the possibility of publication bias (on primary outcomes) using funnel plots of effect estimates on the horizontal axis against their standard errors (on the vertical axis on a reversed scale). We will apply Egger's regression asymmetry test to funnel plots to test for funnel plot asymmetry (Egger 1997).
Data synthesis
Given the overlap between interventions in terms of common components (e.g. relaxation training, social problem solving, addressing cognitive bias), we intend to combine all trials in a single meta‐analysis for each outcome, at each time point, where data are available. We will use both a fixed‐effect and a random‐effects model and compare the results in order to assess the impact of statistical heterogeneity. Unless contraindicated (e.g. by funnel plot asymmetry), we will present the results from the random‐effects model. If we find severe funnel plot asymmetry, we will present both fixed‐effect and random‐effects analyses, based on the assumption that asymmetry indicates that neither model is appropriate. Where both indicate the presence (or absence) of an effect, we will be reassured; and when the models disagree, we will report and discuss both. We will calculate overall effects using inverse variance methods. Where some studies report an outcome as a dichotomous measure and others use a continuous measure, we will convert the results of the former from an odds ratio (OR) to an SMD, as long as we have reason to believe that both are measuring the same construct and assuming that the underlying continuous measure has approximately a normal distribution or logistic distribution. If this is not the case, we will conduct separate analyses.
We will explore other potential sources of heterogeneity in a subgroup analysis (see Subgroup analysis and investigation of heterogeneity). If it is inappropriate to combine the results of any studies in a meta‐analysis, we will describe them narratively.
'Summary of findings' table
We will use the Grades of Recommendations, Assessment, Development and Evaluation (GRADE) approach to create 'Summary of findings' tables when appropriate, and will use GRADE profiler (GRADEPRO) to import data from Review Manager 5 (RevMan 2014) to create 'Summary of findings' tables using the outcomes highlighted in Types of outcome measures and summary statistics outlined in Measures of treatment effect. For details of the GRADE approach and factors that influence the assessment, see Table 1, Table 2, Table 3, and Table 4.
| Underlying methodology | Quality rating |
| Randomised trials; or double‐upgraded observational studies | High |
| Downgraded randomised trials; or upgraded observational studies | Moderate |
| Double downgraded randomised trials; or observational studies | Low |
| Triple downgraded randomised trials; or downgraded observational studies, or case series/case reports | Very low |
Copy of Table 12.2.a from Schünemann 2011.
GRADE: Grades of Recommendation, Assessment, Development and Evaluation.
| 1. Limitations in the design and implementation of available studies suggesting high likelihood of bias |
| 2. Indirectness of evidence (indirect population, intervention, control, outcomes) |
| 3. Unexplained heterogeneity or inconsistency of results (including problems with subgroup analyses) |
| 4. Imprecision of results (wide confidence intervals) |
| 5. High probability of publication bias |
Copy of Table 12.2.b from Schünemann 2011.
| 1. Large magnitude of effect |
| 2. All plausible confounding would reduce a demonstrated effect or suggest a spurious effect when results show no effect |
| 3. Dose‐response gradient |
| Quality of evidence | Explanation |
| High | Further research is very unlikely to change our confidence in the estimate of effect |
| Moderate | Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate |
| Low | Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate |
| Very low | We are very uncertain about the estimate |
GRADE: Grades of Recommendations, Assessment, Development and Evaluation
Subgroup analysis and investigation of heterogeneity
We will explore possible sources of heterogeneity using subgroup analyses based on:
-
gender (male only, female only, mixed groups);
-
age (birth to five years of age and five years of age and over, to reflect the typical age of attendance at first formal education); and
-
level of intervention (universal, indicated prevention, treatment for clinical level problems).
Sensitivity analysis
We will conduct a sensitivity analysis based on risk of bias. Given that it is almost impossible for participants and personnel to be blinded to the intervention condition, we will concentrate on risk of bias relating to sequence generation, allocation concealment and incomplete outcome reporting. We will remove studies deemed at high risk of bias to ascertain the effect of these studies on the pooled estimate.
We will also examine the influence of different procedural decisions taken by the review authors (see Unit of analysis issues and Dealing with missing data) through sensitivity analyses to determine the impact of the decisions on the overall results.
| Underlying methodology | Quality rating |
| Randomised trials; or double‐upgraded observational studies | High |
| Downgraded randomised trials; or upgraded observational studies | Moderate |
| Double downgraded randomised trials; or observational studies | Low |
| Triple downgraded randomised trials; or downgraded observational studies, or case series/case reports | Very low |
| Copy of Table 12.2.a from Schünemann 2011. GRADE: Grades of Recommendation, Assessment, Development and Evaluation. | |
| 1. Limitations in the design and implementation of available studies suggesting high likelihood of bias |
| 2. Indirectness of evidence (indirect population, intervention, control, outcomes) |
| 3. Unexplained heterogeneity or inconsistency of results (including problems with subgroup analyses) |
| 4. Imprecision of results (wide confidence intervals) |
| 5. High probability of publication bias |
| Copy of Table 12.2.b from Schünemann 2011. |
| 1. Large magnitude of effect |
| 2. All plausible confounding would reduce a demonstrated effect or suggest a spurious effect when results show no effect |
| 3. Dose‐response gradient |
| Quality of evidence | Explanation |
| High | Further research is very unlikely to change our confidence in the estimate of effect |
| Moderate | Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate |
| Low | Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate |
| Very low | We are very uncertain about the estimate |
| GRADE: Grades of Recommendations, Assessment, Development and Evaluation | |
