Contacting authors by telephone increased response proportions compared with emailing: results of a randomized study

Abstract

Objective

The aim of the study was to compare response proportions and research costs of telephone calling vs. continued emailing nonresponding authors of studies included in a systematic review.

Study Design and Setting

Key features of included studies were poorly reported in a systematic review of diabetes quality improvement interventions. We developed a survey to request additional information from contact authors. After three email contact attempts, only 76 of 279 authors (27%) had completed the survey. In this study, we randomly assigned nonresponding authors to contact by telephone calling vs. continued emailing to compare the effect of these strategies on response proportions and research costs.

Results

We randomized 87 authors to telephone and 89 to email contact. Telephone contact increased survey completion (36.7% vs. 20.2%; adjusted risk difference of 15.6% [95% confidence interval: 2.90%, 28.4%]; adjusted odds ratio 2.26 [95% confidence interval: 1.10, 4.76]) but required more time to deliver (20 vs. 10 hours in total; 14 vs. 7 minutes per randomized author; 26 vs. 4 weeks), and cost more (total intervention cost of $504 Canadian dollars vs. $252 for the telephone and email arm, respectively).

Conclusion

Contacting nonresponding authors of included studies by telephone increased response compared with emailing but required more investigator time and had higher cost.

Introduction

Publications of primary research studies often do not report enough information about the study methods and results to allow systematic reviewers to assess the evidence in the studies [1], [2]. To address incomplete reporting, reviewers can attempt to contact study authors to obtain additional information, for example, about study characteristics [3], research methods [4], or outcomes [3], [4], [5]. Incomplete reporting may be even more problematic in studies of complex interventions where consensus on intervention terminology is lacking [6], descriptions of interventions are often incomplete [7], [8], and variation in components, as well as the interactions between components and study characteristics influence treatment effects [9], [10].

Existing guidance on the conduct of systematic reviews recommends contacting authors to obtain additional information [3], [4], [5], but the evidence supporting the effectiveness of contacting authors is limited [1], [4], [11]. Five comparative studies identified in a Cochrane review of methods for obtaining unpublished data suggest that authors are more likely to respond to requests for additional information if they pertain to a clarification of study methods (vs. study results), are received by email (vs. letter or fax), and refer to more recently published studies [12]. However, the quality of the five studies identified by the review was poor—only two studies randomly assigned authors to different methods of contact and only one of the three nonrandomized comparative studies had been published in full text [12]. Furthermore, only two studies considered the time and monetary cost associated with conducting author contact [13], [14]. One nonrandomized study assessed time for author response in days (but not investigator time) [13], and another nonrandomized study assessed the cost of intervention delivery, but only for study materials (i.e., postage and paper) [14].

In a recent update of our review of diabetes quality improvement (QI) interventions [15], [16], we found that the included primary studies often provided incomplete information about intervention components, populations, and settings [16], [17]. In preparing to update the review again, the review team (including the methodologists, clinicians, and knowledge users listed in Appendix A) developed a tailored survey to request additional information from the contact authors of all included studies (n = 279) [18]. We sent the Web-based survey once a week via email to corresponding authors until they responded, requested not to be contacted further, or we completed a maximum of three contact attempts. Emails to contact authors were sent from the email address of a senior investigator on our research team (J.M.G.). We provided a financial incentive: authors who completed the survey were entered in a draw for one of five $100 (CAD) gift certificates. A total of 76 authors (27.2%) had completed the survey after three email attempts. In a single occasion, speaking to a primary study author by telephone (to address survey troubleshooting issues) led to a positive interaction and subsequent survey completion. Based on this observation, we hypothesized that attempting to contact nonresponding authors by telephone would increase rates of survey completion compared with continued emailing (i.e., for a fourth time). Given the paucity of evidence about the preferred methods for author contact and the likely higher costs of telephone contact, we decided to evaluate this hypothesis in a randomized trial.