Types of studies

All relevant randomised controlled trials. We excluded quasi‐randomised trials, such as those where allocation is undertaken on surname. If a trial was described as double‐blind, but it was implied it had been randomised, we included these trials in a sensitivity analysis. Randomised cross‐over studies were eligible but only data up to the point of first cross‐over because of the instability of the problem behaviours and the likely carryover effects of all treatments.

As CBT requires the person to actively engage and participate in the therapy it may not be possible to “blind” the participant to condition (that is, it may not be possible to provide a placebo control condition to reduce the effects of anticipated outcome on behalf of the participant). However, it is possible and desirable to “blind” the trialist to condition (that is, the trialist collecting outcome data is unaware of the allocation of the individual participant). Accordingly, “single blind” trials are considered of appropriate methodological quality for the assessment of this type of intervention.

The outcomes of trials that describe a “single‐blind” procedure will be compared with trials that do not describe any “blinding” procedure. If there was no substantive difference within primary outcomes (see Types of outcome measures) when these “non‐blinded studies were added, then they were included in the final analysis. If there was a substantive difference, only single blinded randomised trials were used and the results of the sensitivity analysis are described in the text.

Types of participants

People with a current diagnosis of schizophrenia, diagnosed by any criteria, irrespective of gender or race. We did not include participants who had very late onset of illness (onset after the age of 60 years) or those with other psychotic disorders such as bipolar disorder, manic depressive psychosis, substance‐induced psychosis, significant physical or sensory difficulties or people with coexisting developmental learning difficulties. If studies randomised people with schizophrenia and those with the above disorders we only included trials where more than 50% of the participants had a diagnosis of schizophrenia.

This review will not include trials that report outcomes from participants deemed to be "at‐risk" of developing schizophrenia in the future.

Types of interventions

1. Cognitive behavioural therapy
The label cognitive behavioural therapy has been applied to a variety of interventions, accordingly, is difficult to provide a single, unambiguous definition. Recognising this, the reviewers constructed criteria that were felt to be both workable and to capture the elements of good practice in CBT.

In order to be classified as 'well defined' the intervention must clearly demonstrate the following components:

1. A discrete psychological intervention, which is in addition to, and separate from other therapeutic interventions (for example, behavioural family therapy) and

2. Recipients establish links between their symptoms, thoughts and beliefs, and consequent distress or problem behaviour and

3. The re‐evaluation of their perceptions, beliefs or reasoning relating to the target symptoms.

All therapies that do not meet these criteria (or that provide insufficient information) but are labelled as 'CBT' or 'Cognitive Therapy' were included as “less well defined CBT”. A sensitivity analysis on the primary outcomes of this review (see types of outcomes) will be conducted in order to investigate whether a “well defined” implementation of this therapy presents with differential outcomes.

In addition, a sensitivity analysis will be undertaken between studies that have employed qualified CBT therapists compared with relatively unqualified CBT therapists. Qualified CBT therapists may be defined as

a) Persons possessing appropriate professional qualifications for the provision of CBT (e.g., BABCP accreditation, Diploma in CBT, or other professionally accredited qualifications involving CBT as major part of training (e.g. Clinical or Counselling Psychologist)) or

b) In situations were the qualifications of the therapist are unclear but they appear to have received training in CBT or specific training for the trial and there is a thorough adherence protocol.

Unqualified CBT therapists may be defined as persons not possessing appropriate professional qualifications nor report of training and adherence protocol.

2. Other psychosocial interventions
Where standard care has been supplemented by additional psychological and/or social interventions, such as supportive therapy, psycho‐education, family therapy and other 'talking therapies'.

This review distinguishes between trials that described 'active' psychosocial interventions (e.g., family therapy) aimed at a meaningful symptom reduction and those trials which have used 'non‐active' psychosocial interventions (e.g., unstructured conversations) which act as merely a control for the non‐specific effects of therapy (e.g., time spent with therapist). Outcomes are presented separately for Active and Non‐active psychosocial interventions and the pooled effect of these trials is also presented.

Types of outcome measures

Outcomes can be categorised as of short, medium or long duration. Short‐term outcome is defined as occurring within the period typically associated with active treatment. National Institute for Clinical excellence asserts that “for it to make a difference, [the patient] should have CBT treatment for more than 6 months, meeting for more than ten treatment sessions” (NICE 2009). Accordingly, this review has grouped outcomes into those measured in the short‐term (within 24 weeks of the onset of therapy), medium term (within 24 to 52 weeks of the onset of therapy) and long term (over 52 weeks since the onset of therapy).

Outcomes can be grouped into the following broad areas

1. General functioning: these outcomes relate to meaningful changes in symptomatology and general clinical condition, recovery and well‐being.

2. Mental state: these outcomes refer to the presence or absence of symptoms of psychosis as well as continuous measures relating to the characteristics of such symptoms (e.g. preoccupation; conviction; frequency; duration; intensity, loudness; perceived interference with daily living) and insight. Measures of general affect (e.g. anxiety, depression, shame, hopelessness, anger; self‐esteem) and symptom‐related affect measures (e.g. voice‐related distress; delusional distress) are also considered. The presence or frequency of problematic behaviours (suicide attempts; deliberate self‐harm; violence to others, etc) and functional and adaptive behaviours (e.g. increased coping strategies) are included.

3. Adverse outcomes: All health interventions have the capacity for unintended and unwanted side effects. To date there has been a paucity of studies that have attempted to identify adverse effects of psychological therapies. Such outcomes might include dependency, increased distress, increased family dysfunction and disengagement from mental health services.

4. Service utilisation: The measurement of service utilisation and functional outcomes may convey important information regarding health economic benefits, as well as provide indirect markers of personal independence. Such outcomes might include number of acute hospital /inpatient respite days, number of acute hospital admissions or equivalent (e.g. Home treatment/crisis team intervention; respite admissions), changes in legal status (MHA 1983), changes in level of care (including accommodation type and intensity of service (Assertive Outreach Team versus Community Mental Health Team)).

5. Functional outcomes: These outcomes might include changes in employment, occupational and educational status, level of received benefits or social welfare, perceived quality of life and level of social functioning.

6. Pharmacological treatment: These outcomes would include alterations in the degree of compliance with the prescribed medication regimen, as well as alterations to the prescribed medication including changes in type of medication and prescribed dosage. Unwanted side effects will also be assessed.

7. Economic outcomes: These outcomes would include both the direct costs of CBT (e.g., costs relating to the provision of therapy) and the indirect costs of CBT (e.g., reduction in medication, reduction in relapse, etc).

Electronic searches

1. Electronic searches for update

1.1 The Cochrane Schizophrenia Group's Register (March 2010) was searched using the phrase:

{[(*cogniti* AND (*behavio* or therap*)) OR (*cogniti* and (*technique* or *restructur* or *challeng*)) OR (*self* and (*instruct* or *management* or *attribution*)) OR (*rational* and *emotiv*) in title, abstract, index terms of REFERENCE] or [Cognitive* in interventions of STUDY]}

This register is compiled by systematic searches of major databases, hand searches and conference proceedings (see group module).

For previous searches please see Appendices

Searching other resources

We also searched reference lists of included and excluded studies for additional relevant trials.

Selection of studies

Three reviewers (AM, DH & CAJ) independently inspected all identified citations . When disputes arose as to which category a citation should be allocated, resolution was attempted by discussion. When this was not possible the full article was acquired. Two reviewers (DH, CAJ) independently inspected all articles identified in this way. When disputes arose as to whether an article was indeed relevant to this review, resolution was attempted by discussion. When this was not possible another reviewer (CI) was asked to read the article and decide. IR, AM and CI were sent 30% of the citations and articles, included and excluded by DH and CAJ, to check the use of inclusion criteria. Finally where resolution was not possible because further information was necessary, data were not entered and the trial was allocated to the list of those awaiting assessment and attempts were made to contact authors in order to provide further clarification of data.

Data extraction and management

1. Extraction
DH and CAJ extracted all data from the selected trials, again, working independently of each other. Disputes were resolved by discussion and adjudication from the other reviews (AM, CI and IC) if necessary. When it was not possible to extract data or if further information was needed, attempts were made to contact the authors.

2. Management
Data were extracted onto standard, simple forms.

3. Scale‐derived data
We included continuous data from rating scales only if: (a) the psychometric properties of the measuring instrument had been described in a peer‐reviewed journal (Marshall 2000); (b) the measuring instrument was not written or modified by one of the trialists; (c) the measuring instrument is either (i) a self‐report or (ii) completed by an independent rater or relative (not the therapist).

Assessment of risk of bias in included studies

Again working independently, DH and CAJ assessed risk of bias using the tool described in the Cochrane Collaboration Handbook (Higgins 2008). This tool encourages consideration of how the sequence was generated, how allocation was concealed, the integrity of blinding at outcome, the completeness of outcome data, selective reporting and other biases.

The risk of bias in each domain is categorised as follows:
A. Low risk of bias: plausible bias unlikely to seriously alter the results (categorised as 'Yes' in Risk of Bias table)
B. High risk of bias: plausible bias that seriously weakens confidence in the results (categorised as 'No' in Risk of Bias table)
C. Unclear risk of bias: plausible bias that raises some doubt about the results (categorised as 'Unclear' in Risk of Bias table)

Measures of treatment effect

We adopted p=0.05 as the conventional level of statistical significance but are especially cautious where results were only slightly below this, and we report 95% confidence intervals in preference to p‐values.

1. Dichotomous data
Where possible, efforts were made to convert outcome measures to dichotomous data. This could be done by identifying cut‐off points on rating scales and dividing participants accordingly into 'clinically improved' or 'not clinically improved'. It was generally assumed that if there had been a 50% reduction in a scale‐derived score such as the Brief Psychiatric Rating Scale (BPRS, Overall 1962) or the Positive and Negative Syndrome Scale (PANSS, Kay 1987), this could be considered as a clinically significant response (Leucht 2005a, Leucht 2005b). If data based on these thresholds were not available, we used the primary cut‐off presented by the original authors.

We calculated the relative risk (RR) and its 95% confidence interval (CI) based on the random‐effects model, as this takes into account any differences between studies even if there is no statistically significant heterogeneity. It has been shown that RR is more intuitive (Boissel 1999) than odds ratios and, that odds ratios tend to be interpreted as RR by clinicians (Deeks 2000). This misinterpretation then leads to an overestimate of the impression of the effect.

2. Continuous data
2.1 Summary statistic
For continuous outcomes we estimated a Mean Difference (MD) between groups. MDs were based on the random‐effects model as this takes into account any differences between studies even if there is no statistically significant heterogeneity. We did not calculate Standardised Mean Differences measures.

2.2 Endpoint versus change data
Since there is no principal statistical reason why endpoint and change data should measure different effects (Higgins 2008), we used scale endpoint data, which is easier to interpret from a clinical point of view. If endpoint data were not available, we used change data.

2.3 Skewed data
Continuous data on clinical and social outcomes are often not normally distributed. To avoid the pitfall of applying parametric tests to non‐parametric data, we applied the following standards to all data before inclusion: (a) standard deviations and means were reported in the paper or obtainable from the authors; (b) when a scale starts from the finite number zero, the standard deviation, when multiplied by two, is less than the mean (as otherwise the mean is unlikely to be an appropriate measure of the centre of the distribution, (Altman 1996); (c) if a scale starts from a positive value (such as PANSS which can have values from 30 to 210) the calculation described above were modified to take the scale starting point into account. In these cases skew is present if 2SD>(S‐S min), where S is the mean score and S min is the minimum score. Endpoint scores on scales often have a finite start and end point and these rules can be applied. When continuous data were presented on a scale, which includes a possibility of negative values (such as change data), it is difficult to tell whether data were skewed or not. Skewed data from studies of less than 200 participants were entered in additional tables rather than into an analysis.

Skewed data pose less of a problem when looking at means if the sample size is large and were entered into syntheses.

Unit of analysis issues

1. Cluster trials
Studies increasingly employ 'cluster randomisation' (such as randomisation by clinician or practice) but analysis and pooling of clustered data poses problems. Firstly, authors often fail to account for intraclass correlation in clustered studies, leading to a 'unit of analysis' error (Divine 1992) whereby p values are spuriously low, confidence intervals unduly narrow and statistical significance overestimated. This causes type I errors (Bland 1997; Gulliford 1999).

Where clustering is not accounted for in primary studies, we presented data in a table, with a (*) symbol to indicate the presence of a probable unit of analysis error. In subsequent versions of this review we will seek to contact first authors of studies to obtain intraclass correlation coefficient of their clustered data and to adjust for this by using accepted methods (Gulliford 1999). Where clustering had been incorporated into the analysis of primary studies, we present these data as if from a non‐cluster randomised study, but adjusted for the clustering effect.

We have sought statistical advice and have been advised that the binary data as presented in a report should be divided by a 'design effect'. This is calculated using the mean number of participants per cluster (m) and the intraclass correlation coefficient (ICC) [Design effect = 1+(m‐1)*ICC] (Donner 2002). If the ICC was not reported it was assumed to be 0.1 (Ukoumunne 1999).

If cluster studies has been appropriately analysed taking into account intraclass correlation coefficient and relevant data documented in the report, synthesis with other studies would have been possible using the generic inverse variance technique.

2. Cross‐over trials
A major concern of cross‐over trials is the carry‐over effect. It occurs if an effect (e.g. pharmacological, physiological or psychological) of the treatment in the first phase is carried over to the second phase. As a consequence on entry to the second phase the participants can differ systematically from their initial state despite a wash‐out phase. For the same reason cross‐over trials are not appropriate if the condition of interest is unstable (Elbourne 2002). As both effects are very likely in schizophrenia, we will only use data of the first phase of cross‐over studies.

3. Studies with multiple treatment groups
Where a study involved more than two treatment arms, if relevant, the additional treatment arms were presented in comparisons. Where the additional treatment arms were not relevant, these data were not reproduced.

Dealing with missing data

1. Overall loss of credibility
At some degree of loss of follow up data must lose credibility (Xia 2009). We are forced to make a judgment where this is for the very short‐term trials likely to be included in this review. Should more than 40% of data be unaccounted for by 12 weeks we did not reproduce these data or use them within analyses.

2. Binary
In the case where attrition for a binary outcome is between 0 and 40% and outcomes of these people are described, we included these data as reported. Where these data were not clearly described, data were presented on a 'once‐randomised‐always‐analyse' basis, assuming an intention to treat analysis. Those lost to follow‐up were all assumed to have a negative outcome, with the exception of the outcome of death. For example, for the outcome of relapse, those who were lost to follow‐up all relapsed. A final sensitivity analysis was undertaken testing how prone the primary outcomes were to change when 'completed' data only were compared to the intention to treat analysis using the negative assumption.

3. Continuous
In the case where attrition for a continuous outcome is between 0 and 40% and completer‐only data were reported, we have reproduced these.

4. Intention‐to‐treat (ITT)
Intention‐to‐treat was used when available. We anticipated that in some studies, in order to do an ITT analysis, the method of last observation carried forward (LOCF) would be employed within the study report. As with all methods of imputation to deal with missing data, LOCF introduces uncertainty about the reliability of the results. Therefore, where LOCF data have been used in the analysis, it was indicated in the review.

Assessment of heterogeneity

1. Clinical heterogeneity
We considered all included studies hoping to use all studies together. Should clear unforeseen issues have been apparent that may add obvious clinical heterogeneity, we noted these issues, considered them in analyses and undertook sensitivity analyses for the primary outcome.

2. Statistical
2.1 Visual inspection
We visually inspected graphs to investigate the possibility of statistical heterogeneity.

2.2 Employing the I‐squared (I²) statistic
Heterogeneity between studies was investigated by considering the I‐squared (I²) method alongside the chi² 'p' value. The I² provides an estimate of the percentage of inconsistency thought to be due to chance (Higgins 2003). The importance of the observed value of I² depends on (a) magnitude and direction of effects and (b) strength of evidence for heterogeneity (e.g. 'p' value from chi² test, or a confidence interval for I²). I² estimate greater than or equal to 50% accompanied by a statistically significant chi² statistic, was interpreted as evidence of substantial levels of heterogeneity (Section 9.5.2 ‐ Higgins 2008) and reasons for heterogeneity were explored. If the inconsistency was high and the clear reasons were found, data were presented separately.

Assessment of reporting biases

Reporting biases arise when the dissemination of research findings is influenced by the nature and direction of results (Egger 1997). These are described in section 10.1 of the Cochrane Handbook (Higgins 2008). We are aware that funnel plots may be useful in investigating reporting biases but are of limited power to detect small‐study effects. We did not use funnel plots for outcomes where there were ten or fewer studies, or where all studies were of similar sizes. In other cases, where funnel plots were possible, we sought statistical advice in their interpretation.

Data synthesis

Where possible we employed a random effects model for analyses. We understand that there is no closed argument for preference for use of fixed or random‐effect models. The random‐effects method incorporates an assumption that the different studies are estimating different, yet related, intervention effects. This does seem true to us, however, random‐effects does put added weight onto the smaller of the studies ‐ those trials that are most vulnerable to bias.

Subgroup analysis and investigation of heterogeneity

1. Subgroup analysis
We plan one subgroup analysis. We are interested whether people in their first episode of illness differ in their outcomes with those who have a longer history of illness. Should data be available we plan to present the information from these groups of people separately for the primary outcome or the nearest equivalent outcome that the relevant studies report.

2. Investigation of heterogeneity
If data are clearly statistically heterogeneous we first checked that data were correctly extracted and entered and that we had made no unit of analysis errors. If the high levels of heterogeneity remained we did not undertake a meta‐analysis at this point for if there is considerable variation in results, and particularly if there is inconsistency in the direction of effect, it may be misleading to quote an average value for the intervention effect. Instead we would have explored possible sources of heterogeneity. We do not pre‐specify any characteristics of studies that may be associated with heterogeneity except those relating to the quality of trial method. If no clear association could be shown by sorting studies by quality of methods a random‐effects meta‐analysis was performed. Should another characteristic of the studies be highlighted by the investigation of heterogeneity, perhaps some clinical heterogeneity not hitherto predicted but plausible causes of heterogeneity, these post‐hoc reasons will be discussed and the data analysed and presented. However, should the heterogeneity be substantially unaffected by use of random‐effects meta‐analysis and no other reasons for the heterogeneity be clear, the results of the individual trials would be presented without a meta‐analysis.

Sensitivity analysis

Where data permitted, sensitivity analysis was undertaken in order to see if sub‐grouping the data resulted in important changes in the results. Three such sub‐groupings were pre‐specified: (i) rigorous criteria for diagnosing schizophrenia as opposed to more loose criteria; (ii) rigorous criteria for describing cognitive behavioural therapy as opposed to more loose diagnosis; and (iii) people in first episode of illness versus those at a later stage of illness.