Variations in the reporting of outcomes used in systematic reviews of treatment effectiveness research in bladder pain syndrome

Abstract

This paper investigates the quality of outcomes reported in systematic reviews and randomised controlled trials (RCTs) of bladder pain syndrome and its relationship with study quality and journal impact factor. We searched until August 2013 the Cochrane Library, EMBASE, Medline, CINAHL, LILACS and SIGLE, without language restrictions. Quality of outcome reporting in systematic reviews and constituent RCTs was assessed using a 6-point scale. Overall study quality was assessed using the AMSTAR and Jadad scoring systems, and impact factor in the year of publication was noted. Spearman's rank correlation was calculated. There were 8 systematic reviews, with a total of 28 RCTs (1732 patients), reporting 5 outcomes using 19 different measurement scales. The outcomes reported in individual RCTs were urinary symptoms (100%), pain (64%), quality of life (39%), general wellbeing (36%) and bladder capacity (36%). The mean quality of outcomes reported was 1.63 (95% CI 0.29–2.96) for systematic reviews and 3.25 (95% CI 2.80–3.70) for RCTs. The quality of outcomes reported showed correlation with overall study quality (0.90, 95% CI 0.79–0.95, p < 0.0001) but not with journal impact factor (0.07, 95% CI −0.31–0.43, p = 0.35). Multivariable linear regression showed a relationship between quality of outcome reporting and study quality (β = 0.05, p < 0.0001), adjusting for effects of study type, impact factor and journal type. There is a need to generate consensus over a set of core outcomes in bladder pain syndrome using standardised reporting tools and to disseminate these through good publication practice.

Introduction

Treatment effectiveness studies examine changes in outcomes. Inconsistencies in reported outcomes and the tools used to measure these, with missing outcome data and outcome reporting bias, are often seen across studies [1]. Consistency in outcomes is essential to allow direct comparison of effects. Inconsistency hinders evidence syntheses, limiting their usefulness with downstream negative impact on care quality. The use of core outcomes is required to improve the translation of evidence into practice [2].

We wanted to examine if quality of outcome reporting was linked to other publication features. Bladder pain syndrome (BPS) (formerly known as interstitial cystitis and painful bladder syndrome), a common condition associated with considerable disability [3], [4], has trials and reviews evaluating various treatments to achieve symptomatic control. These outcomes are measured using a range of scales and scores. Any chronic condition would serve as a good exemplar to empirically address our questions, but we chose BPS as this condition is of particular interest to the authors who are assessing the evidence on efficacy of treatments in BPS. We acknowledge the lack of understanding around the aetiology of this condition and consensus on diagnosing and managing it, despite recent guidelines from the American Urological Association [5].

We systematically identified primary and secondary outcomes and assessed the variation in diversity and quality of outcome measures used to evaluate treatments for BPS in published systematic reviews and their constituent trials. We evaluated the relationship of quality of outcomes reported with overall study quality and journal impact factor in a controlled analysis adjusting for the effects of year of publication, commercial funding, study design and journal type.