Types of studies

All relevant randomised controlled trials. We excluded quasi‐randomised trials, such as those where allocation is undertaken on surname. If a trial was described as double‐blind, but it was implied it had been randomised, we included these trials in a sensitivity analysis.

Randomised cross‐over studies will be eligible but only data up to the point of first cross‐over because of the instability of the problem behaviours and the likely carry‐over effects of all treatments (Elbourne 2002).

Types of participants

People with a current diagnosis of schizophrenia or other similar serious psychotic illness.

We did not include participants who had very late onset of illness (onset after the age of 60 years) or those with other psychotic disorders such as bipolar disorder, manic depressive psychosis, substance induced psychosis or people with coexisting developmental learning difficulties, significant physical or sensory difficulties. If studies randomised people with schizophrenia and those with the above disorders we only included trials where more than 50% of the participants had a diagnosis of schizophrenia.

Types of interventions

1. Cognitive Behavioural Therapy: Cognitive Behavioural Therapy as an additional treatment to standard care. We defined CBT as a discrete psychological intervention where

(i) recipients establish links between their thoughts, feelings or actions with respect to the current or past symptoms, and/or functioning; and
(ii) the re‐evaluation of their perceptions, beliefs or reasoning relate to the target symptoms.

In addition, a further component of the intervention should involve the following:

(i) recipients monitor their own thoughts, feelings or behaviours with respect to the symptom or recurrence of symptoms; and/or
(ii) the promotion of alternative ways of coping with the target symptom; and/or
(iii) the reduction of distress; and/or
(iv) the improvement of functioning.

2. Specific pharmacological interventions (for example additional adjunct medications or substances such as ethyl‐EPA (fish oil) which are thought to enhance standard medication) which are an additional treatment to standard care.

Types of outcome measures

All outcomes are grouped as short term (within 12 weeks of the onset of therapy), medium term (within 13 to 26 weeks of the onset of therapy) or long term (over 26 weeks since the onset of therapy).

Electronic searches

We searched The Cochrane Schizophrenia Group Trials Register (February 2009) using the phrase:

{[(*cogniti* AND (*behavio* or therap*)) OR (*cogniti* and (*technique* or *restructur* or *challeng*)) OR (*self* and (*instruct* or *management* or *attribution*)) OR (*rational* and *emotiv*) in title, abstract, index terms of REFERENCE] or [Cognitive* in interventions of STUDY]}

This register is compiled by systematic searches of major databases, hand searches and conference proceedings (see Group Module).

Searching other resources

We also searched reference lists of included and excluded studies for additional relevant trials.

Selection of studies

We divided the search citations into two lists, the authors CJ and CI inspected one list independently and the authors DH and AM inspected the other list independently. We identified potentially relevant reports and ordered full papers for reassessment. Where difficulties or disputes arose we asked another author, IC, for help and if it was impossible to decide, those full papers were ordered for assessment. This process was repeated for the full papers. If it was impossible to resolve disagreements these studies were added to those awaiting assessment and the authors of the papers contacted for clarification. IC was sent ten per cent of the citations and articles, included and excluded by all authors, to check the use of inclusion criteria. Non‐concurrence in trial selection was reported.

Data extraction and management

1. Extraction
Authors CJ and CI independently extracted data from included studies on their list while DH and AM independently extracted data from included studies on their list. Again, any disagreement was discussed, decisions documented and, if necessary, authors of studies were contacted for clarification. With remaining problems IC helped clarify issues and those final decisions were documented.

2. Management
Data were extracted onto standard, simple forms.

3. Scale‐derived data
We included continuous data from rating scales only if: (a) the psychometric properties of the measuring instrument had been described in a peer‐reviewed journal (Marshall 2000); (b) the measuring instrument was not written or modified by one of the trialists; (c) the measuring instrument is either (i) a self‐report or (ii) completed by an independent rater or relative (not the therapist).

Assessment of risk of bias in included studies

Again working independently, CJ, CI, DH and AM assessed risk of bias using the tool described in the Cochrane Collaboration Handbook (Higgins 2008). This tool encourages consideration of how the sequence was generated, how allocation was concealed, the integrity of blinding at outcome, the completeness of outcome data, selective reporting and other biases.

The risk of bias in each domain is categorised as follows:

A. Low risk of bias: plausible bias unlikely to seriously alter the results (categorised as 'Yes' in Risk of Bias table)
B. High risk of bias: plausible bias that seriously weakens confidence in the results (categorised as 'No' in Risk of Bias table)
C. Unclear risk of bias: plausible bias that raises some doubt about the results (categorised as 'Unclear' in Risk of Bias table)

We did not include trials with high risk of bias (at least three out of five domains categorised as 'No') in the meta‐ analysis. If the authors disagreed, the final rating was made by consensus with our fifth reviewer IC. If a trial was difficult to categorise due to inadequate details of randomisation and other characteristics, we contacted the authors of the studies in order to obtain further information.

Measures of treatment effect

1. Dichotomous data.
Where possible, efforts were made to convert outcome measures to dichotomous data. This could be done by identifying cut‐off points on rating scales and dividing participants accordingly into 'clinically improved' or 'not clinically improved'. It was generally assumed that if there had been a 50% reduction in a scale‐derived score such as the Brief Psychiatric Rating Scale (BPRS, Overall 1962) or the Positive and Negative Syndrome Scale (PANSS, Kay 1987), this could be considered as a clinically significant response (Leucht 2005a, Leucht 2005b). If data based on these thresholds were not available, we used the primary cut‐off presented by the original authors.

We calculated the relative risk (RR) and its 95% confidence interval (CI) based on the random‐effects model, as this takes into account any differences between studies even if there is no statistically significant heterogeneity. It has been shown that RR is more intuitive (Boissel 1999) than odds ratios and that odds ratios tend to be interpreted as RR by clinicians (Deeks 2000). This misinterpretation then leads to an overestimate of the impression of the effect. When the overall results were significant we calculated the number needed to treat to provide benefit (NNTB) and the number needed to treat to induce harm (NNTH) as the inverse of the risk difference

2. Continuous data.
2.1 Summary statistic
For continuous outcomes we estimated a Weighted Mean Difference (WMD) between groups. The WMD calculations were based on the random‐effects model as this takes into account any differences between studies even if there is no statistically significant heterogeneity. We did not calculate Standardised Mean Difference measures.

2.2 Endpoint versus change data
Since there is no principal statistical reason why endpoint and change data should measure different effects (Higgins 2008), we used scale endpoint data, which is easier to interpret from a clinical point of view. If endpoint data were not available, we used changed data.

2.3 Skewed data
Continuous data on clinical and social outcomes are often not normally distributed. To avoid the pitfall of applying parametric tests to non‐parametric data, we applied the following standards to all data before inclusion: (a) standard deviations and means are reported in the paper or obtainable from the authors; (b) when a scale starts from the finite number zero, the standard deviation, when multiplied by two, is less than the mean (as otherwise the mean is unlikely to be an appropriate measure of the centre of the distribution, (Altman 1996); (c) if a scale starts from a positive value (such as PANSS which can have values from 30 to 210) the calculation described above will be modified to take the scale starting point into account. In these cases skew is present if 2SD>(S‐S min), where S is the mean score and S min is the minimum score. Endpoint scores on scales often have a finite start and end point and these rules can be applied. When continuous data are presented on a scale, which includes a possibility of negative values (such as change data), it is difficult to tell whether data are skewed or not. Skewed data from studies of less than 200 participants were entered in additional tables rather than into an analysis. Skewed data pose less of a problem when looking at means if the sample size is large and were entered into syntheses.

Unit of analysis issues

1. Cluster trials
Studies increasingly employ 'cluster randomisation' (such as randomisation by clinician or practice) but analysis and pooling of clustered data poses problems. Firstly, authors often fail to account for intraclass correlation in clustered studies, leading to a 'unit of analysis' error (Divine 1992) whereby p values are spuriously low, confidence intervals unduly narrow and statistical significance overestimated. This causes type I errors (Bland 1997, Gulliford 1999).

Where clustering is not accounted for in primary studies, we presented data in a table, with a (*) symbol to indicate the presence of a probable unit of analysis error. In subsequent versions of this review we will seek to contact first authors of studies to obtain intraclass correlation coefficients for their clustered data and to adjust for this by using accepted methods (Gulliford 1999). Where clustering had been incorporated into the analysis of primary studies, we present these data as if from a non‐cluster randomised study, but adjusted for the clustering effect.

We have sought statistical advice and have been advised that the binary data as presented in a report should be divided by a 'design effect'. This is calculated using the mean number of participants per cluster (m) and the intraclass correlation coefficient (ICC) [Design effect = 1+(m‐1)*ICC] (Donner 2002). If the ICC was not reported it was assumed to be 0.1 (Ukoumunne 1999).

If cluster studies has been appropriately analysed taking into account intraclass correlation coefficients and relevant data documented in the report, synthesis with other studies would have been possible using the generic inverse variance technique.

2. Cross‐over trials
A major concern of cross‐over trials is the carry‐over effect. It occurs if an effect (e.g. pharmacological, physiological or psychological) of the treatment in the first phase is carried over to the second phase. As a consequence on entry to the second phase the participants can differ systematically from their initial state despite a wash‐out phase. For the same reason cross‐over trials are not appropriate if the condition of interest is unstable (Elbourne 2002). As both effects are very likely in schizophrenia, we will only use data of the first phase of cross‐over studies.

3. Studies with multiple treatment groups
Where a study involved more than two treatment arms, if relevant, the additional treatment arms were presented in comparisons. Where the additional treatment arms were not relevant, these data were not reproduced.

Dealing with missing data

1. Overall loss of credibility
At some degree of loss of follow‐up data must lose credibility (Xia 2007). We are forced to make a judgment where this is for the short‐term trials likely to be included in this review. Should more than 40% of data be unaccounted for we did not reproduce these data or use them within analyses.

2. Binary
In the case where attrition for a binary outcome is between 0 and 40% and outcomes of these people are described, we included these data as reported. Where these data were not clearly described, data were presented on a 'once‐randomised‐always‐analyse' basis, assuming an intention to treat analysis. Those lost to follow‐up were all assumed to have a negative outcome, with the exception of the outcome of death. For example, for the outcome of relapse, those who were lost to follow‐up all relapsed. A final sensitivity analysis was undertaken testing how prone the primary outcomes were to change when 'completed' data only were compared to the intention to treat analysis using the negative assumption.

3. Continuous
In the case where attrition for a continuous outcome is between 0 and 40% and completer‐only data were reported, we have reproduced these.

4. Intention‐to‐treat (ITT)
Intention‐to‐treat (ITT) was used when available. We anticipated that in some studies, in order to do an ITT analysis, the method of last observation carried forward (LOCF) would be employed within the study report. As with all methods of imputation to deal with missing data, LOCF introduces uncertainty about the reliability of the results. Therefore, where LOCF data have been used in the analysis, it was indicated in the review.

Assessment of heterogeneity

1. Clinical heterogeneity
We considered all included studies hoping to use all studies together. Should clear unforeseen issues have been apparent that may add obvious clinical heterogeneity, we noted these issues, considered them in analyses and undertook sensitivity analyses for the primary outcome.

2. Statistical
2.1 Visual inspection
We visually inspected graphs to investigate the possibility of statistical heterogeneity.

2.2 Employing the I‐squared statistic
Heterogeneity between studies was investigated by considering the I‐squared method alongside the chi2 'p' value. The I² provides an estimate of the percentage of inconsistency thought to be due to chance (Higgins 2003. The importance of the observed value of I² depends on (i) magnitude and direction of effects and (ii) strength of evidence for heterogeneity (e.g. 'p' value from chi² test, or a confidence interval for I²). I² estimate greater than or equal to 50% accompanied by a statistically significant chi² statistic, was interpreted as evidence of substantial levels of heterogeneity (Section 9.5.2 ‐ Higgins 2008) and reasons for heterogeneity were explored. If the inconsistency was high and clear reasons were found, data were presented separately.

Assessment of reporting biases

Reporting biases arise when the dissemination of research findings is influenced by the nature and direction of results (Egger 1997). These are described in section 10.1 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2008). We are aware that funnel plots may be useful in investigating reporting biases but are of limited power to detect small‐study effects. We did not use funnel plots for outcomes where there were ten or fewer studies, or where all studies were of similar sizes. In other cases, where funnel plots were possible, we sought statistical advice in their interpretation.

Data synthesis

Where possible we employed a fixed‐effect model for analyses. We understand that there is no closed argument for preference for use of fixed or random‐effects models. The random‐effects method incorporates an assumption that the different studies are estimating different, yet related, intervention effects. This does seem true to us, however, random‐effects does put added weight onto the smaller of the studies ‐ those trials that are most vulnerable to bias. For this reason we favour using fixed‐effect models employing random‐effects only when investigating heterogeneity.

Subgroup analysis and investigation of heterogeneity

If data are clearly heterogeneous we checked that data are correctly extracted and entered and that we had made no unit of analysis errors. If the high levels of heterogeneity remained we did not undertake a meta‐analysis at this point for if there is considerable variation in results, and particularly if there is inconsistency in the direction of effect, it may be misleading to quote an average value for the intervention effect. We would have wanted to explore heterogeneity. We pre‐specify no characteristics of studies that may be associated with heterogeneity except quality of trial method. If no clear association could be shown by sorting studies by quality of methods a random‐effects meta‐analysis was performed. Should another characteristic of the studies be highlighted by the investigation of heterogeneity, perhaps some clinical heterogeneity not hitherto predicted but plausible causes of heterogeneity, these post‐hoc reasons will be discussed and the data analysed and presented. However, should the heterogeneity be substantially unaffected by use of random‐effects meta‐analysis and no other reasons for the heterogeneity be clear, the final data were presented without a meta‐analysis.

Sensitivity analysis

We planned sensitivity analyses a priori for examining the change in the robustness of the sensitivity to including studies with implied randomisation (see Criteria for considering studies for this review: Types of studies), skewed and non‐skewed data, inappropriate comparator doses of drug and different clinical groups ‐ the latter being defined post hoc. If inclusion of studies with implied randomisation made no substantive difference to the primary outcome they were left in the final analyses. For outcomes with both skewed data and non‐skewed data, we investigated the effect of combining all data together and if no substantive difference was noted then the potentially skewed data were left in the analyses. We recognise that we may not have considered some clinical causes for heterogeneity that become more obvious after seeing the data. The sensitivity of the primary outcome of including or removing groups of trials with distinct clinical groups will be investigated post hoc. We are fully aware that these are weak investigations and only generate and do not prove hypotheses. We do not anticipate this last sensitivity analysis but wish to leave the potential for investigating any omission we may have made in consideration of studies at the stage of writing the protocol.