Primary outcomes

Secondary outcomes

1. 1. The Cochrane Schizophrenia Group Trials Register

The CSzG Trials Register was searched (May 2008, February 2013) using the phrase:

[((*peric?azin* or *aolept* or *nemactil* or *neulactil* or *neulept?l* or *neuperil* or propericiazine or RP 8909 or SKF 20716) in title, abstract and index fields in REFERENCE) OR (*peric?azin* or propericiazine or RP 8909 or SKF 20716) in interventions field in STUDY]

This register is compiled by systematic searches of major databases, handsearches and conference proceedings (see Group Module).

1. Extraction

Review author HEM extracted data from all included studies. In addition, to ensure reliability, MQA independently extracted data from a random sample of these studies, comprising 10% of the total. Again, any disagreement was discussed and decisions documented. With remaining problems SM helped clarify issues and these final decisions were documented. Data presented only in graphs and figures were extracted whenever possible, but included only if two review authors independently had the same result. We had intended to contact authors through an open‐ended request in order to obtain missing information or for clarification whenever necessary but as all included studies were completed over 30 years ago, this was not attempted. If studies were multi‐centre, where possible, we extracted data relevant to each component centre separately.

2. Management

1. Binary data

For binary outcomes, we calculated a standard estimation of the risk ratio (RR) and its 95% confidence interval (CI). It has been shown that RR is more intuitive (Boissel 1999) than odds ratios and that odds ratios tend to be interpreted as RR by clinicians (Deeks 2000). The Number Needed to Treat/Harm (NNT/H) statistic with its confidence intervals is intuitively attractive to clinicians but is problematic both in its accurate calculation in meta‐analyses and interpretation (Hutton 2009). For binary data presented in the 'Summary of findings' tables, where possible, we calculated illustrative comparative risks.

2. Continuous data

We were unable to extract any continuous data for this review. If we had found such data, we would have estimated mean difference (MD) between groups. We prefer not to calculate effect size measures (standardised mean difference SMD). In future updates, if scales of very considerable similarity are used, we will presume there is a small difference in measurement, and we will calculate effect size and transform the effect back to the units of one or more of the specific instruments.

1. Cluster trials

Studies increasingly employ 'cluster randomisation' (such as randomisation by clinician or practice) but analysis and pooling of clustered data poses problems. Firstly, authors often fail to account for intra‐class correlation in clustered studies, leading to a 'unit of analysis' error (Divine 1992) whereby P values are spuriously low, confidence intervals unduly narrow and statistical significance overestimated. This causes type I errors (Bland 1997; Gulliford 1999).

Our search did not reveal any cluster‐randomised trials, otherwise, for these kind of data, where clustering is not accounted for in primary studies, we would have presented data in a table, with a (*) symbol to indicate the presence of a probable unit of analysis error. If we find cluster‐randomised trial data in subsequent versions of this review, we will seek to contact first authors of studies to obtain intra‐class correlation coefficients (ICCs) for their clustered data and to adjust for this by using accepted methods (Gulliford 1999). If clustering has been incorporated into the analysis of primary studies, we will present these data as if from a non‐cluster randomised study, but adjust for the clustering effect.

We sought statistical advice during the protocol state of this review and have been advised that the binary data as presented in a report should be divided by a 'design effect'. This is calculated using the mean number of participants per cluster (m) and the ICC [Design effect = 1+(m‐1)*ICC] (Donner 2002). If the ICC is not reported, it will be assumed to be 0.1 (Ukoumunne 1999).

If cluster studies have been appropriately analysed taking into account ICCs and relevant data documented in the report, synthesis with other studies will be possible using the generic inverse variance technique.

2. Cross‐over trials

A major concern of cross‐over trials is the carry‐over effect. It occurs if an effect (e.g. pharmacological, physiological or psychological) of the treatment in the first phase is carried over to the second phase. As a consequence, on entry to the second phase the participants can differ systematically from their initial state despite a wash‐out phase. For the same reason, cross‐over trials are not appropriate if the condition of interest is unstable (Elbourne 2002). Both effects are very likely in severe mental illness. We did not identify any studies of this type but only planned to use data from the first phase of cross‐over trials.

3. Studies with multiple treatment groups

All of the included studies in this review had parallel treatment arms. In future updates, where a study involves more than two treatment arms, if relevant, the additional treatment arms will be presented in comparisons. If data are binary they will simply be added and combined within the two‐by‐two table. If data are continuous we will combine data following the formula in section 7.7.3.8  (Combining groups) of the Cochrane Handbook for Systemic reviews of Interventions (Higgins 2011). Where the additional treatment arms are not relevant, we will not use these data.

1. Overall loss of credibility

At some degree of loss of follow‐up, data must lose credibility (Xia 2009). Most of the studies included in this review either had no missing data or the missing data were of an acceptable proportion of the total (< 50%). For future updates, we choose that, for any particular outcome, should more than 50% of data be unaccounted for, we will not reproduce these data or use them within the analyses. If, however, more than 50% of those in one arm of a study are lost, but the total loss is less than 50%, this will be addressed within the 'Summary of findings' tables by down‐rating quality. Finally, we will also downgrade quality within the 'Summary of findings' tables should loss be 25% to 50% in total.

2. Binary

In the case where attrition for a binary outcome is between 0% and 50% and where these data are not clearly described, we presented data on a 'once‐randomised‐always‐analyse' basis (an intention‐to‐treat analysis). Those leaving the study early are all assumed to have the same rates of negative outcome as those who completed, with the exception of the outcome of death and adverse effects. For these outcomes, the rate of those who stay in the study ‐ in that particular arm of the trial ‐ will be used for those who did not. We planned to undertake a sensitivity analysis to test how prone the primary outcomes were to change when data only from people who completed the study to that point are compared to the intention‐to treat analysis using the above assumptions, however, we did not identify studies with the appropriate data to enable us to do this.

3. Continuous

1. Clinical heterogeneity

We considered all included studies initially, without seeing comparison data, to judge clinical heterogeneity. We simply inspected all studies for clearly outlying people or situations which we had not predicted would arise. We did not find any clinical heterogeneity amongst the included studies.

2. Methodological heterogeneity

We considered all included studies initially, without seeing comparison data, to judge methodological heterogeneity. We simply inspected all studies for clearly outlying methods which we had not predicted would arise. We did not identify any methodological outliers in the included studies.

3. Statistical heterogeneity

1. Protocol versus full study

Reporting biases arise when the dissemination of research findings is influenced by the nature and direction of results. These are described in section 10.1 of the Cochrane Handbook for Systemic reviews of Interventions (Higgins 2011). We tried to locate protocols of included randomised trials. If the protocol was available, we planned to compare outcomes in the protocol and in the published report. However, the studies included in this particular review were conducted before the initiation of clinical trial registration, therefore, the protocols were not available. Thus, we compared outcomes listed in the methods section of the trial report with actually reported results.

2. Funnel plot

Reporting biases arise when the dissemination of research findings is influenced by the nature and direction of results (Egger 1997). These are again described in Section 10 of the Cochrane Handbook for Systemic reviews of Interventions (Higgins 2011). We are aware that funnel plots may be useful in investigating reporting biases but are of limited power to detect small‐study effects. We did not use funnel plots for outcomes as there are less than 10 included studies. In future updates, where funnel plots are possible, we will seek statistical advice in their interpretation.

1. Subgroup analyses

2. Investigation of heterogeneity

We did not find any unacceptable level of inconsistency when combining data for this review. However, for future updates, if inconsistency is high, this will be reported. First, we will investigate whether data have been entered correctly. Second, if data are correct, we will visually inspect the graph and remove outlying studies to see if homogeneity is restored. Should this occur with data contributing to the summary finding of no more than around 10% of the total weighting, data will be presented. If not, data will not be pooled and issues will be discussed. We know of no supporting research for this 10% cut‐off but are investigating use of prediction intervals as an alternative to this unsatisfactory state.

When unanticipated clinical or methodological heterogeneity are obvious, we will simply state hypotheses regarding these for future reviews or versions of this review. We do not anticipate undertaking analyses relating to these.

1. Implication of randomisation

We did not identify any trials with implied randomisation in this review. For future updates, we aim to include trials in a sensitivity analysis if they are described in some way as to imply randomisation. For the primary outcomes, we will include these studies and if there is no substantive difference when the implied randomised studies are added to those with better description of randomisation, then all data will be employed from these studies.

2. Assumptions for lost binary data

We did not have to make assumptions with people lost to follow‐up in this review, but for future updates, where assumptions have to be made regarding people lost to follow‐up (see Dealing with missing data), we will compare the findings of the primary outcomes when we use our assumptions and when we use data only from people who complete the study to that point. If there is a substantial difference, we will report results and discuss them but will continue to employ our assumption.

For future updates, where assumptions have to be made regarding missing SDs data (see Dealing with missing data), we will compare the findings of the primary outcomes when we use our assumptions and when we use data only from people who complete the study to that point. A sensitivity analysis will be undertaken to test how prone results are to change when completer‐only data only are compared with the imputed data using the above assumption. If there is a substantial difference, we will report results and discuss them, but will continue to employ our assumption.

3. Risk of bias

We had planned to analyse the effects of excluding trials that are judged to be at high risk of bias across one or more of the domains of randomisation (implied as randomised with no further details available), allocation concealment, blinding and outcome reporting for the meta‐analysis of the primary outcome. However, we were unable to perform this comparison due to the fact that only limited data were available for primary outcomes. For instance, there were only two studies contributed data on global state, both had high risk rating on one of the above domains. In future updates, if there are more data available, we hope to carry out the sensitivity analysis. Data from trials with high risk of bias ratings will only be included in the analysis, if the exclusion of trials at high risk of bias does not substantially alter the direction of effect or the precision of the effect estimates.

4. Imputed values

We also planned to undertake a sensitivity analysis to assess the effects of including data from trials where we used imputed values for ICC in calculating the design effect in cluster‐randomised trials, however, we did not impute any data in this version of the review.

Should this become the case in future updates, if substantial differences are noted in the direction or precision of effect estimates in any of the sensitivity analyses listed above, we will not pool data from the excluded trials with the other trials contributing to the outcome, but present them separately.

5. Fixed and random effects

All data were synthesised using a fixed‐effect model, however, we also synthesised data for the primary outcome using a random‐effects model to evaluate whether this alters the significance of the results.