Keywords
Core outcome set, COS, study within a trial, SWAT, infant feeding, childhood obesity, response rates, questionnaire completion, outcome measurements.
-
Views
59
Downloads
How to cite this article
Griffin C, Toomey E, Queally M et al. Influence of providing information to participants about development of trial outcomes on response rates and attitudes to questionnaire completion: Protocol for a study within a trial [version 1; peer review: 1 approved, 1 approved with reservations]. HRB Open Res 2019, 2:2 (https://doi.org/10.12688/hrbopenres.12895.1)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Export Citation
Sciwheel
EndNote
Ref. Manager
Bibtex
ProCite
Sente
▬
✚
Study Protocol
Influence of providing information to participants about development of trial outcomes on response rates and attitudes to questionnaire completion: Protocol for a study within a trial
[version 1; peer review: 1 approved, 1 approved with reservations]
PUBLISHED 09 Jan 2019
OPEN PEER REVIEW
REVIEWER STATUS
This article is included in the HRB-TMRN gateway.
This article is included in the Public and Patient Involvement collection.
Abstract
Corresponding author:
Karen Matvienko-Sikar
Competing interests:
No competing interests were disclosed.
Grant information:
This work was supported by the Health Research Board Interdisciplinary Capacity Enhancement Award (HBR ICE-2015-1026) and the HRB Trials Methodology Research Network SWAT Award 2017.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Copyright:
© 2019 Griffin C et al.
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
How to cite: Griffin C, Toomey E, Queally M et al. Influence of providing information to participants about development of trial outcomes on response rates and attitudes to questionnaire completion: Protocol for a study within a trial [version 1; peer review: 1 approved, 1 approved with reservations]. HRB Open Res 2019, 2:2 (https://doi.org/10.12688/hrbopenres.12895.1)
First published: 09 Jan 2019, 2:2 (https://doi.org/10.12688/hrbopenres.12895.1)
Latest published: 20 May 2019, 2:2 (https://doi.org/10.12688/hrbopenres.12895.2)
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
There is a newer version
of this article available.
of this article available.
Introduction
Evaluation of questionnaire responses is an important dimension of the critical appraisal of health research (Edwards et al., 2009). Incomplete questionnaire responses and participant attrition increases the likelihood of bias and reduces statistical power in trials through reduction of the effective sample size (Edwards et al., 2009; Fewtrell et al., 2008; Schulz & Grimes, 2002). Further, even in well-designed studies, factors related to research management can influence participant retention and impact questionnaire response rates, leading to research waste (Al-Shahi Salman et al., 2014).
A number of potential methods to effectively increase response rates have been identified, including: the use of monetary incentives, (Brueton et al., 2014); telephone or postal contact with participants prior to questionnaire distribution (Edwards et al., 2009); and personalising questionnaires or survey packs with participants name and/or including a hand-written signature from the principal investigator (Sahlqvist et al., 2011; Scott & Edwards, 2006). However, reviews have highlighted heterogeneity among strategies used across trials (e.g. differences in the types of incentives used between studies) thus limiting synthesis and conclusions that can be drawn about effectiveness (Edwards et al., 2009). As such, there remains a need to further examine strategies to improve response rates.
Improving communication with participants about aspects related to their trial participation may be one useful strategy. Such communication is posited to enhance participant engagement with research processes in ways that are meaningful to the participant (Gillies & Entwistle, 2012). For instance, there is some evidence to suggest that participants who feel they are better informed about trial processes tend to have more favourable attitudes toward the trial and are therefore more willing to participate in the trial (Ellis et al., 2001). However, participants and the general public are suggested to have a poor understanding of different aspects of health research (Ellis et al., 1999). This is problematic if it influences participant attitudes and limits engagement with trial processes; there is therefore scope to improve information provision to trial participants.
In terms of enhancing communication to improve questionnaire response rates specifically, one approach may be via providing participants with information about how outcomes are chosen and/or questionnaires are developed for use in trials. This would be particularly useful where outcomes and questionnaires are developed via engagement with expert stakeholders as is typically done in the development of Core Outcome Sets (COS)(Williamson et al., 2017). COSs are standardised sets of outcomes that represent the minimum outcomes that should be measured and reported in trials for a specific health area or population (Williamson et al., 2012; Williamson et al., 2017). COSs improve evidence synthesis by reducing outcome heterogeneity and reporting risk of bias (Williamson et al., 2012), which have been noted in a range of health areas, including paediatrics (Gardner & Kelleher, 2017; Webbe et al., 2018), infant feeding (Whitford et al., 2018), and childhood obesity (Matvienko-Sikar et al., 2018; Redsell et al., 2016). Expert stakeholders in COS development can include patients, clinicians, trialists, researchers and the public (Williamson et al., 2017). It is suggested that engagement with such stakeholders increases the likelihood that a COS will be relevant and used by these stakeholders in research and practice (Williamson et al., 2017); however, how engagement of participant stakeholders influences subsequent participant endorsement and use of the COS has not been fully examined. Knowledge about stakeholder involvement in COS development may influence participant attitudes and response rates via perceptions of subjective norms around the importance of trial outcomes, where stakeholders included are representatives of the participant group. Where COS development involved clinicians and/or practitioners, such stakeholders may be perceived by patient and/or public participants as representing credible sources. This may enhance participant response rates as credible sources have been identified as a useful behaviour change technique (BCT) (Michie et al., 2013), for increasing trial engagement in other trials (Nyman et al., 2018; Parveen et al., 2016; Redfern et al., 2016).
Providing participants with information about COS in trials serves a dual purpose by informing participants about the outcomes of importance being measured in the trial, and highlighting the role of relevant stakeholders in developing the COS being measured in the trial. The influence of informing participants about the development and use of COS in trials on their attitudes towards and completion of trial questionnaires has not yet been examined. This research posits that including information related to COS development and measurement may serve to increase participant knowledge of these processes and/or lead to more favourable attitudes toward questionnaire completion, which would subsequently increase response rates. The aim of this study is therefore to conduct a study within a trial (SWAT) (Treweek et al., 2018a) to examine if provision of information regarding development of a COS influences participants questionnaire response rates and attitudes towards questionnaire completion.
Methods
This SWAT is registered on the Northern Ireland Hub for Trials Methodology Research SWAT Repository (SWAT57).
Design
This is a randomised, single-blinded, parallel group intervention SWAT embedded within the Choosing Healthy Eating for Infant Health (Cherish) feasibility trial (protocol currently in preparation for submission). The Cherish trial involves a brief clinical intervention targeting parents and caregivers to improve infant feeding behaviours between the ages of 0–13 months, delivered during routine primary care-based vaccination visits, alongside an implementation strategy targeted at the healthcare professional (HCP) level to support the delivery of this clinical intervention.
Study participants
Participants for the SWAT will be the parents or primary caregivers participating in the Cherish feasibility trial. The Cherish trial participants are recruited from all parents or primary caregivers of infants under 6 weeks of age attending vaccination visits with a participating GP and or practice nurse in the trial site, a primary care centre in the south of Ireland.
The Study Within A Trial (SWAT)
The SWAT intervention is a written informational intervention, consisting of a brief written explanation about the COS used in the development of the Cherish feasibility trial questionnaires, including the involvement of relevant stakeholders in developing this COS. The COS used is a COS of infant feeding outcomes for inclusion in trials of infant feeding interventions to prevent childhood obesity (Matvienko-Sikar et al., 2018a; Matvienko-Sikar et al., 2018b). The COS was developed in a four-stage process, involving expert stakeholders in the final three stages (Matvienko-Sikar et al., 2017a). Expert stakeholders were parents, HCPs, researchers and childcare professionals. Participants will be randomly assigned using a random number generator to one of two conditions (I1 or I2) and will be blinded (single blind) to the condition assigned (Figure 1). In the I1 condition, participants will receive the SWAT intervention, in the form of the brief COS information. The intervention will be presented prior to completing the Cherish and SWAT questionnaires at trial baseline (when the infant is less than 2 months old). Participants randomised to the I2 condition will receive the information on COS following completion of both the Cherish and SWAT questionnaires at baseline. The questionnaires will be completed by participants online, at home or over the phone with a Cherish researcher, dependent on participant preference. The Cherish study questionnaire consists of all questions relevant for the feasibility trial; the SWAT questionnaire is a brief separate questionnaire, outlined below (Figure 1).
Figure 1. SWAT intervention flowchart.
Information about the COS will be provided to a random sample of half of all participants in a brief paragraph including the following (see Extended data for further information):
A statement that the questionnaires include measurement of outcomes from an infant feeding COS.
A lay-summary of what a COS is and how COSs can improve examination of trial outcomes informed by the Core Outcome Measures in Effectiveness Trials (COMET) Initiative COS lay summary (COMET, 2018).
A brief description of how the infant feeding COS was developed with experts, including parents of infants and HCPs.
Outcome measurement
A 4-item SWAT questionnaire will be included at the end of the Cherish trial questionnaire in the I1 condition; a 2-item SWAT questionnaire will be included at the end of the Cherish trial questionnaire in the I2 condition.
1. The infant-feeding related questions in the questionnaire were useful for gaining insight into how you feed your child.
2. The infant-feeding related questions in the questionnaire were appropriate for gaining insight into how you feed your child
3. The information provided about Core Outcome Sets (COSs) influenced my completion of the questionnaire.
4. How did the information about Core Outcome Sets (COSs) influence your completion of the questionnaire?
The I1 condition will complete all four questions. The I2 condition will complete the first two SWAT questionnaire items only. Questions 1 to 3 are closed-ended, with responses rated on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree’. The single open-ended question (Question 4) allows participants to describe in their own words how the information on COS influenced their completion of the questionnaire (Extended data for formatted SWAT questionnaire).
Analysis
Quantitative analysis: All questionnaire data will be entered into SPSS Version 24 software. Questionnaire response rates will be calculated for each of the intervention conditions, including completion of the full questionnaire and individual item response rates. Chi squared tests will compare the proportion of the questionnaire completed for the two conditions. Potential differences between participant baseline characteristics will also be examined and should differences be observed, these will be controlled for.
Qualitative analysis: Responses to the open-ended question will be entered into NVivo 12 for qualitative data management and will be analysed using thematic analysis following Braun and Clarke (2006) guidelines. However, if there is insufficient detail in the open-ended responses then they will be examined narratively.
Discussion
The SWAT embedded within the Cherish feasibility trial is an important step for evaluating additional potential benefits of COSs in trial methodology beyond the benefits of COSs for evidence synthesis (Williamson et al., 2012). Evidence suggests that more well-informed participants are more willing to participate and engage in health research (Ellis et al., 2002; Treweek et al., 2018b). Increasing participant knowledge of different aspects of trial processes therefore has the potential to increase response rates and minimise attrition in trials. Specifically, informing trial participants about the development and measurement of a COS has the potential to increase participant response rates in a number of ways. The first is through provision of information to increase participant knowledge. The second is through highlighting subjective norms in terms of outcomes of importance in trials where a COS developed by stakeholder groups representative of the participant group; subjective norms can influence behavioural intentions and subsequent behaviour. The third is via use of credible sources, for instance in the form of perceived expert stakeholders in COS development.
Some weaknesses of this SWAT need to be considered. For instance, recruitment of participants for the SWAT is dependent on numbers recruited to the larger Cherish trial. Precise and accurate conclusions can only be drawn from an appropriate sample size, therefore insufficient sample size will adversely impact statistical power to detect a difference between the two conditions (Nayak, 2010). A strength of this SWAT is that it is embedded within a larger trial conducted within an engaged primary care practice. Recruitment will be conducted by post and in-person in the primary care practice, thus maximising and utilizing all avenues for participant recruitment and engagement. This SWAT uses a mixed-methods approach to data collection, with closed ended questions allowing for evaluation of participants attitudes towards questionnaire completion in both conditions. The open-ended question allows participants in condition I1 to describe in their own words how COS information influenced their completion of the questionnaire. This approach will facilitate an understanding of whether and how the SWAT intervention worked (Farquhar et al., 2011). A further strength is that this SWAT was designed following best practice SWAT guidelines (Treweek et al., 2018a), particularly in terms of appropriate use of randomisation and appropriate planning of analysis and implementation. Furthermore, this SWAT draws on mechanisms of behaviour including, BCTs (Michie et al., 2013) and the theory of planned behaviour (Ajzen, 1991). These theoretical underpinnings ensure that the proposed rationale of this SWAT moves beyond simply thinking that the information alone will influence questionnaire completion rates (Nyman et al., 2018). Through examining if informing participants of the use of an infant feeding COS effects questionnaire response rates and questionnaire completion, findings of the SWAT will significantly contribute to the literature on strategies for maximising participant response rates in trials.
Ethics approval and consent to participate
The research was approved in Ireland by the Clinical Research Ethics Committee of the Cork Teaching Hospitals, UCC.
On commencement of the trial, all participants will provide signed consent for participation in the study and publication of results.
Data availability
Underlying data
Currently there is no available data associated with this article as the feasibility trial has yet to commence.
Extended data
Open Science Framework: COS SWAT (The SWAT Questionnaire), https://doi.org/10.17605/OSF.IO/VHJS4 (Matvienko-Sikar, 2018).
Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).
Comments on this article Comments (1)
Version 2
VERSION 2
PUBLISHED 20 May 2019
Revised
Version 1
VERSION 1
PUBLISHED 09 Jan 2019
Discussion is closed on this version, please comment on the latest version above.
-
Discussion is closed on this version, please comment on the latest version above.
Download
Export To
CITE
how to cite this article
Griffin C, Toomey E, Queally M et al. Influence of providing information to participants about development of trial outcomes on response rates and attitudes to questionnaire completion: Protocol for a study within a trial [version 1; peer review: 1 approved, 1 approved with reservations] HRB Open Res 2019, 2:2 (https://doi.org/10.12688/hrbopenres.12895.1)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.
Open Peer Review
Current Reviewer Status:
Key to Reviewer Statuses
VIEW
HIDE
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations
A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 09 Jan 2019
Views
22
How to cite this report:
Treweek SP, Goulao B and Innes K. Reviewer Report For: Influence of providing information to participants about development of trial outcomes on response rates and attitudes to questionnaire completion: Protocol for a study within a trial [version 1; peer review: 1 approved, 1 approved with reservations]. HRB Open Res 2019, 2:2 (https://doi.org/10.21956/hrbopenres.13964.r26467)
The direct URL for this report is:
https://hrbopenresearch.org/articles/2-2/v1#referee-response-26467
https://hrbopenresearch.org/articles/2-2/v1#referee-response-26467
NOTE: it is important to ensure the information in square brackets after the title is included in this citation.
Reviewer Report 06 Feb 2019
Shaun P. Treweek, Health Services Research Unit (HSRU), Institute of Applied Health Sciences, School of Medicine, Medical Sciences and Nutrition, University of Aberdeen, Aberdeen, UK
Beatriz Goulao, Health Services Research Unit (HSRU), Institute of Applied Health Sciences, School of Medicine, Medical Sciences and Nutrition, University of Aberdeen, Aberdeen, UK
Approved with Reservations
VIEWS 22
SWATs such as this are a good way to improve the evidence base for trial process decision-making, so we welcome this protocol. We do have a few comments, all of which are about reporting clarity.
General
... Continue reading
General
... Continue reading
CITE
HOW TO CITE THIS REPORT
Treweek SP, Goulao B and Innes K. Reviewer Report For: Influence of providing information to participants about development of trial outcomes on response rates and attitudes to questionnaire completion: Protocol for a study within a trial [version 1; peer review: 1 approved, 1 approved with reservations]. HRB Open Res 2019, 2:2 (https://doi.org/10.21956/hrbopenres.13964.r26467)
The direct URL for this report is:
https://hrbopenresearch.org/articles/2-2/v1#referee-response-26467
https://hrbopenresearch.org/articles/2-2/v1#referee-response-26467
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
26
How to cite this report:
Dodd S. Reviewer Report For: Influence of providing information to participants about development of trial outcomes on response rates and attitudes to questionnaire completion: Protocol for a study within a trial [version 1; peer review: 1 approved, 1 approved with reservations]. HRB Open Res 2019, 2:2 (https://doi.org/10.21956/hrbopenres.13964.r26468)
The direct URL for this report is:
https://hrbopenresearch.org/articles/2-2/v1#referee-response-26468
https://hrbopenresearch.org/articles/2-2/v1#referee-response-26468
NOTE: it is important to ensure the information in square brackets after the title is included in this citation.
Reviewer Report 18 Jan 2019
Approved
VIEWS 26
This is a well-written protocol describing a SWAT which will provide useful information regarding the impact of COS knowledge on participant engagement in questionnaire completion. I am happy to recommend that this protocol should be indexed, providing the following issues
... Continue reading
CITE
HOW TO CITE THIS REPORT
Dodd S. Reviewer Report For: Influence of providing information to participants about development of trial outcomes on response rates and attitudes to questionnaire completion: Protocol for a study within a trial [version 1; peer review: 1 approved, 1 approved with reservations]. HRB Open Res 2019, 2:2 (https://doi.org/10.21956/hrbopenres.13964.r26468)
The direct URL for this report is:
https://hrbopenresearch.org/articles/2-2/v1#referee-response-26468
https://hrbopenresearch.org/articles/2-2/v1#referee-response-26468
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Comments on this article Comments (1)
Version 2
VERSION 2
PUBLISHED 20 May 2019
Revised
Version 1
VERSION 1
PUBLISHED 09 Jan 2019
Discussion is closed on this version, please comment on the latest version above.
-
Author Response 20 May 2019Karen Matvienko-Sikar, School of Public Health, University College Cork, Cork, T12 XF62, Ireland20 May 2019Author ResponseReviewer 1: Susanna Dodd
This is a well-written protocol describing a SWAT which will provide useful information regarding the impact of COS knowledge on participant engagement in questionnaire completion. I am ... Continue reading -
Discussion is closed on this version, please comment on the latest version above.
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations -
A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
Competing Interests Policy
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Examples of 'Non-Financial Competing Interests'
- Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper.
- You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors.
- You are a close professional associate of any of the authors (e.g. scientific mentor, recent student).
- You work at the same institute as any of the authors.
- You hope/expect to benefit (e.g. favour or employment) as a result of your submission.
- You are an Editor for the journal in which the article is published.
Examples of 'Financial Competing Interests'
- You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements.
- You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors.
- You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on.
Stay Updated
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Register with HRB Open Research
Already registered? Sign in
This is a well-written protocol describing a SWAT which will provide useful information regarding the impact of COS knowledge on participant engagement in questionnaire completion. I am ... Continue reading Reviewer 1: Susanna Dodd
This is a well-written protocol describing a SWAT which will provide useful information regarding the impact of COS knowledge on participant engagement in questionnaire completion. I am happy to recommend that this protocol should be indexed, providing the following issues are addressed:
Reviewer comment 1. I have only one major comment relating to the apparent omission of details relating to the sample size target. Please could the authors provide details of the recruitment target, along with justification for the required sample size.
Author response 1. The host trial is a feasibility trial, and the SWAT will be limited by the number of participants recruited to the host trial. The following statement has been included in the manuscript in relation to the potential number of participants for recruitment: On average, 450 infants per annum are born to parents attending the primary care centre and during the 3 month recruitment period, it is anticipated that approximately 112 of these will be eligible for recruitment.
Thus given the nature of the host trial and SWAT a sample size calculation is not included in the body of the text. However, based on aiming to detect a moderate effect size, with an error probability of .05 and 80% power, a total sample size of 102 would be required to detect difference between the intervention and control group.
I have a few minor suggestions for improving the grammar and general flow of the article:
Reviewer comment 2. Abstract Methods: 2nd sentence: "consisting" should be replaced with "consists"
Author response 2. This has been changed.
Reviewer comment 3. Introduction: 1st paragraph: 2nd line:
"increases" should be replaced with "increase"
"reduces" should be replaced with "reduce"
Author response 3. These have been changed
Reviewer comment 4. Introduction: 4th paragraph:
2nd line: Add "how" before "questionnaires"
11th line: Move "however" to after "has not"
Author response 4. These have been changed
Reviewer comment 5. Introduction: last paragraph: 7th line:
Add apostrophe after "participants"
Author response 5. This has been changed
Reviewer comment 6.Methods:
SWAT section:
1st paragraph: 11th sentence: Change "dependent" to "depending"
2nd paragraph: 1st sentence: Change this sentence to
"The SWAT informational intervention will be provided to those randomised to the l1 condition and will consist of a brief paragraph..."
Author response 6. These have been changed
Reviewer comment 7
Analysis: Quantitative analysis:
Clarify the final sentence by adding "using logistic regression" after "controlled for".
Author response 7. This has been added
Reviewer comment 8
Discussion: 1st paragraph:
2nd line: Remove "more" before "well-informed"
7th line: Add "has been" after "where a COS"
Author response 8. these changes have been made
Reviewer comment 9
Discussion: 2nd paragraph:
7th line: Add apostrophe at the end of "participants"
11th line: Remove comma after "including"
14th line: Replace "Through" with "By"; replace "if" with "whether"; repace "effects" with "affects"
Data availability: change "is" to "are"
Author Response 9. These changes have been made.
Reviewer 2: Shawn Treweek, Beatriz Goulao, Karen Innes
SWATs such as this are a good way to improve the evidence base for trial process decision-making, so we welcome this protocol. We do have a few comments, all of which are about reporting clarity.
General
Reviewer comment 1. We found the ‘I1 condition’ and ‘I2 condition’ terminology confusing. ‘Condition’ in particular made us think about an illness or disease. We would suggest ‘SWAT intervention’ and ‘SWAT comparator’ but the authors might think of something better. Either way, we’d like to see a change from ‘I1 condition’ and ‘I2 condition’ to something else.
Author Response 1. I1 and I2 have been changed to SWAT Intervention and SWAT Comparator respectively.
Reviewer comment 2. We would have liked to have seen the information about COS that is presented to participants as part of the SWAT, together with the comparator text. Knowing this will make it easier for others to replicate the evaluation of the SWAT intervention. There was a statement suggesting that this was in the extended data but we didn’t find it. Sorry if there was a problem with the information we received rather than what you submitted.
Author Response 2. The following has now been added to the manuscript for clarity of SWAT information presented:
The included SWAT Intervention text is as follows:
This questionnaire includes questions about infant feeding that were put together as part of a core outcome set. Core outcomes sets are a group of outcomes (related to questions in a questionnaire) that should be measured in all studies in a health area. They are important because they allow researchers to bring together findings from many different studies to give us a better understanding about what works and what doesn’t. This improves the quality of information and helps us develop and examine better healthcare programmes and strategies.
Parents of infants, healthcare professionals, researchers, and childcare professionals decided the questions included in this questionnaire as part of the core outcome set process. This means the questions have been decided by people, including parents like you, to help us best measure how people feed their babies.
Abstract
Reviewer comment 1. The abstract says that response rates will be measured as proportion of full questionnaire completion – what happens to questionnaire that are partially completed? Are they counted as non-responses?
Author Response 1. Questionnaire that are returned partially completed will also be examined for proportion of questionnaire completion, and so are not counted as non-responses. The term ‘full questionnaire completion’ was intended to mean of the overall questionnaire, rather than the individual items. The term ‘full’ has now been removed from the abstract and main text.
Methods
Reviewer Comment 1. It would be good to know a few things about the host trial: 1) how many are involved in the feasibility trial? 2) How many items are collected on the host trial questionnaire?
Author Response 1.
1) The feasibility trial is currently recruiting participants and so the number of participants cannot be provided. However, approximately 450 infants per annum are born to parents attending the trial host site. During the 3 month recruitment period, it is anticipated that approximately 112 of these will be eligible for recruitment.
2) The host trial questionnaire includes 91 individual items, with 64 of these items forming 6 scales (e.g. Perceived stress scale).
Reviewer Comment 2. The paper is missing a clear definition of the outcomes it will measure to assess the effect of the SWAT intervention. In particular what is the primary outcome and when will it be measured? We began to speculate as to whether the measurement was only at baseline, which didn’t seem to make sense. What we’d like to see is a clear indication of when the outcome assessment for the SWAT will be done.
Author Response 2. A clear explanation of the SWAT outcomes is now included in the manuscript as outlined below.
Outcome measurement
The primary outcome of the interest is the proportion of questionnaire completion. Individual item response rates will also be assessed in terms of completion of questions on infant feeding outcomes, and other outcomes such as healthcare utilization and parent well-being. This is because the SWAT Intervention text specifically refers to infant feeding, and so this study will examine whether the intervention influenced completion of these outcomes specifically.
A secondary outcome of interest is participant attitudes about questionnaire completion. Data on participant attitudes will be collected via questions included at the end of the CHErIsH questionnaire; the CHErIsH questionnaire can be completed online, in-person, or by phone based on participant preference, and so the SWAT questions can be similarly completed. Quantitative data for participant attitudes will be collected for all participants using the following two questionnaire items, which are rated from on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree’:
1. The infant-feeding related questions in the questionnaire were useful for gaining insight into how you feed your child.
2. The infant-feeding related questions in the questionnaire were appropriate for gaining insight into how you feed your child
Participants in the SWAT Intervention group will also be asked the following two questions, the first of which is closed-ended and rated from on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree’. The second SWAT Intervention question is a single open-ended question that allows participants to describe in their own words how the information on COS influenced their completion of the questionnaire. Both questions are as follows:
3. The information provided about Core Outcome Sets (COSs) influenced my completion of the questionnaire.
4. How did the information about Core Outcome Sets (COSs) influence your completion of the questionnaire?
Reviewer comment 3. We didn’t find Figure 1 helpful. It suggests that both groups are receiving the SWAT intervention but the text in the manuscript says ‘Information about the COS will be provided to a random sample of half of all participants in a brief paragraph..’, which suggests that not everyone gets it. The diagram would be better as a more standard CONSORT-esque figure with the timings as you work you way down the figure linked to the timing of outcome assessment.
Author Response 3. Figure 1 has been revised, as below. The SWAT Intervention group receive the COS information prior to questionnaire completion. The SWAT comparator receive this information once they have completed all questions, to provide a ‘debrief’ and ensure equal information provision for all participants. The SWAT is completed at baseline of the host trial only to determine effects on questionnaire completion and attitudes at time of information presentation, rather than effects over time. The decision to examine the effects of the COS information at baseline only is also guided by resource and time constraints.
Reviewer comment 4. The 4-item ‘SWAT questionnaire’ confused us a bit because not all questions are asked of all participants. Only questions 3 and 4 are unique to the SWAT intervention, we’re guessing that questions 1 and 2 are not specifically linked to the COS text that forms the SWAT intervention. Why not have questions 1 and 2 (common to both arms) as part of the main questionnaire and the SWAT questionnaire is then just questions 3 and 4? We also weren’t sure how the items on the questionnaire related to the outcomes of the SWAT. They are not response rates so we guess that they will be linked to attitudes. It would be good to know how this work will be done with the 5-point Likert scales and how the free-text in question 4 will be handled. Author response 4. The presentation of the ‘swat questionnaire’ has been revised, such that it is made clearer that two questions are asked of all participants, while two questions are only asked of the SWAT Intervention group. Questions 1 and 2 are directly related to the SWAT intervention text (which has now been included in the manuscript for clarity) as they relate directly to the infant feeding outcomes. These questions have also now been presented more clearly as addressing a secondary aim of the SWAT, which is to assess participant attitudes of questionnaire completion. Further information on how data will be analysed is now also included. The revised text is as follows:
Outcome measurement
The primary outcome of interest is the proportion of questionnaire completion. Individual item response rates will also be assessed in terms of completion of questions on infant feeding outcomes, and other outcomes such as healthcare utilization and parent well-being. This is because the SWAT Intervention text specifically refers to infant feeding, and so this study will examine whether the intervention influenced completion of these outcomes specifically.
A secondary outcome of interest is participant attitudes about questionnaire completion. Data on participant attitudes will be collected via questions included at the end of the CHErIsH questionnaire; the CHErIsH questionnaire can be completed online, in-person, or by phone based on participant preference, and so the SWAT questions can be similarly completed. Quantitative data for participant attitudes will be collected for all participants using the following two questionnaire items, which are rated from on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree’:
1. The infant-feeding related questions in the questionnaire were useful for gaining insight into how you feed your child.
2. The infant-feeding related questions in the questionnaire were appropriate for gaining insight into how you feed your child
Participants in the SWAT Intervention group will also be asked the following two questions, the first of which is closed-ended and rated from on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree’. The second SWAT Intervention question is a single open-ended question that allows participants to describe in their own words how the information on COS influenced their completion of the questionnaire. Both questions are as follows:
3. The information provided about Core Outcome Sets (COSs) influenced my completion of the questionnaire.
4. How did the information about Core Outcome Sets (COSs) influence your completion of the questionnaire?
Analysis
Quantitative analysis: All questionnaire data will be entered into SPSS Version 24 software. Questionnaire response rates will be calculated for each of the SWAT Intervention and SWAT Comparator including proportion of completion of the CHErIsH questionnaire and individual item response rates. Chi squared tests will compare the proportion of the questionnaire completed for the two conditions. Potential differences between participant baseline characteristics (age, sex, education) will also be examined and should differences be observed, these will be controlled for.
Data from the SWAT Intervention group in response to the question 3 will be descriptively summarised in terms of participants’ mean attitude rating, standard deviation and range of ratings. As this data is only collected from the SWAT Intervention, inferential statistics will not be conducted.
Qualitative analysis: Responses to the open-ended question will be entered into NVivo 12 for qualitative data management and will be analysed using thematic analysis following Braun and Clarke (2006) guidelines. This will involve an iterative process of reading and re-reading the data, developing initial line codes, followed by categorisation and development of themes. However, if there is insufficient detail in the open-ended responses then they will be examined narratively.
Reviewer comment 5. Where is the SWAT outcome assessment done? Is it a postal questionnaire, linked to a visit for the host trial, or something else?
Author Response 5. The four SWAT questions are included at the end of the host trial questionnaire. As participants in the host trial can complete the questionnaire online, in-person or by phone, the SWAT questions can also be completed in this way. The following has been added to the text to clarify: the CHErIsH questionnaire can be completed online, in-person, or by phone based on participant preference, and so the SWAT questions can be similarly completed.
Reviewer 6. We weren’t clear why the researchers were measuring response rates for full questionnaire and individual items and which of these is considered the most important? And we assumed (but weren’t sure) that the completion of the host trial’s primary outcome was the trial outcome that you were concentrating on with regard to increasing response rates. Or is it all of the host trial outcomes?
Author response 6. This SWAT is measuring overall proportion of response rates and response rates for individual items, with both considered equally important in the context of this study. This is because core outcome sets (COS) are not typically the only outcomes to be measured in a trial, though they are the recommended minimum, so it is important to distinguish if the effects of COS provision extends to all outcomes, or just those directly related to the COS (for instance in this case, the infant feeding outcomes).
Reviewer comment 7. It would be useful to see a statement about how many people you think will be involved in the SWAT. This will be limited by the size of the CHErIsH trial so we’re not asking for a sample size calculation, just an indication of how many people are likely to be involved.
Author Response 7. The following text has been added to the paper: On average, 450 infants per annum are born to parents attending the primary care centre and during the 3 month recruitment period, It is anticipated that approximately 112 of these will be eligible for recruitment.
Reviewer comment 8. Can you say a bit more about why (and what) you will adjust for when looking at baseline imbalance. Any differences will be due to chance if the randomisation works though if the sample is small, it is true that these may lead to under or over-estimates of effect if the differences are things that affect your outcomes.
Author response 8. Participant age, sex, education will be controlled for as these have an impact on the study outcomes and therefore may impact on questionnaire completion rates. Adjusting for these variables will also be useful, as the reviewers say, because the potentially small sample size may lead to inaccurate estimation of effects.
Discussion
Reviewer comment 1. The SWAT does involve extra data collection (not all SWATs do) and we could speculate that this could reduce response rates to the host trial questionnaire. Could the authors comment on this?
Author Response 1. Yes, this SWAT includes three additional closed-ended questions (only 2 of which are asked in the SWAT Comparator group) and one open-ended question. The following has been added to the discussion to highlight that this is a potential weakness but that care has been taken to minimise the effects of this:
Inclusion of additional SWAT questions at the end of the CHErIsH trial questionnaire may also increase participant burden, which could impact on questionnaire completion rates. However, care was taken by the research team to develop questions that are as brief as possible to minimise this, and these questions are presented at end of CHErIsH questionnaire such that they their presence will potentially have minimal impact on completion.
This is a well-written protocol describing a SWAT which will provide useful information regarding the impact of COS knowledge on participant engagement in questionnaire completion. I am happy to recommend that this protocol should be indexed, providing the following issues are addressed:
Reviewer comment 1. I have only one major comment relating to the apparent omission of details relating to the sample size target. Please could the authors provide details of the recruitment target, along with justification for the required sample size.
Author response 1. The host trial is a feasibility trial, and the SWAT will be limited by the number of participants recruited to the host trial. The following statement has been included in the manuscript in relation to the potential number of participants for recruitment: On average, 450 infants per annum are born to parents attending the primary care centre and during the 3 month recruitment period, it is anticipated that approximately 112 of these will be eligible for recruitment.
Thus given the nature of the host trial and SWAT a sample size calculation is not included in the body of the text. However, based on aiming to detect a moderate effect size, with an error probability of .05 and 80% power, a total sample size of 102 would be required to detect difference between the intervention and control group.
I have a few minor suggestions for improving the grammar and general flow of the article:
Reviewer comment 2. Abstract Methods: 2nd sentence: "consisting" should be replaced with "consists"
Author response 2. This has been changed.
Reviewer comment 3. Introduction: 1st paragraph: 2nd line:
"increases" should be replaced with "increase"
"reduces" should be replaced with "reduce"
Author response 3. These have been changed
Reviewer comment 4. Introduction: 4th paragraph:
2nd line: Add "how" before "questionnaires"
11th line: Move "however" to after "has not"
Author response 4. These have been changed
Reviewer comment 5. Introduction: last paragraph: 7th line:
Add apostrophe after "participants"
Author response 5. This has been changed
Reviewer comment 6.Methods:
SWAT section:
1st paragraph: 11th sentence: Change "dependent" to "depending"
2nd paragraph: 1st sentence: Change this sentence to
"The SWAT informational intervention will be provided to those randomised to the l1 condition and will consist of a brief paragraph..."
Author response 6. These have been changed
Reviewer comment 7
Analysis: Quantitative analysis:
Clarify the final sentence by adding "using logistic regression" after "controlled for".
Author response 7. This has been added
Reviewer comment 8
Discussion: 1st paragraph:
2nd line: Remove "more" before "well-informed"
7th line: Add "has been" after "where a COS"
Author response 8. these changes have been made
Reviewer comment 9
Discussion: 2nd paragraph:
7th line: Add apostrophe at the end of "participants"
11th line: Remove comma after "including"
14th line: Replace "Through" with "By"; replace "if" with "whether"; repace "effects" with "affects"
Data availability: change "is" to "are"
Author Response 9. These changes have been made.
Reviewer 2: Shawn Treweek, Beatriz Goulao, Karen Innes
SWATs such as this are a good way to improve the evidence base for trial process decision-making, so we welcome this protocol. We do have a few comments, all of which are about reporting clarity.
General
Reviewer comment 1. We found the ‘I1 condition’ and ‘I2 condition’ terminology confusing. ‘Condition’ in particular made us think about an illness or disease. We would suggest ‘SWAT intervention’ and ‘SWAT comparator’ but the authors might think of something better. Either way, we’d like to see a change from ‘I1 condition’ and ‘I2 condition’ to something else.
Author Response 1. I1 and I2 have been changed to SWAT Intervention and SWAT Comparator respectively.
Reviewer comment 2. We would have liked to have seen the information about COS that is presented to participants as part of the SWAT, together with the comparator text. Knowing this will make it easier for others to replicate the evaluation of the SWAT intervention. There was a statement suggesting that this was in the extended data but we didn’t find it. Sorry if there was a problem with the information we received rather than what you submitted.
Author Response 2. The following has now been added to the manuscript for clarity of SWAT information presented:
The included SWAT Intervention text is as follows:
This questionnaire includes questions about infant feeding that were put together as part of a core outcome set. Core outcomes sets are a group of outcomes (related to questions in a questionnaire) that should be measured in all studies in a health area. They are important because they allow researchers to bring together findings from many different studies to give us a better understanding about what works and what doesn’t. This improves the quality of information and helps us develop and examine better healthcare programmes and strategies.
Parents of infants, healthcare professionals, researchers, and childcare professionals decided the questions included in this questionnaire as part of the core outcome set process. This means the questions have been decided by people, including parents like you, to help us best measure how people feed their babies.
Abstract
Reviewer comment 1. The abstract says that response rates will be measured as proportion of full questionnaire completion – what happens to questionnaire that are partially completed? Are they counted as non-responses?
Author Response 1. Questionnaire that are returned partially completed will also be examined for proportion of questionnaire completion, and so are not counted as non-responses. The term ‘full questionnaire completion’ was intended to mean of the overall questionnaire, rather than the individual items. The term ‘full’ has now been removed from the abstract and main text.
Methods
Reviewer Comment 1. It would be good to know a few things about the host trial: 1) how many are involved in the feasibility trial? 2) How many items are collected on the host trial questionnaire?
Author Response 1.
1) The feasibility trial is currently recruiting participants and so the number of participants cannot be provided. However, approximately 450 infants per annum are born to parents attending the trial host site. During the 3 month recruitment period, it is anticipated that approximately 112 of these will be eligible for recruitment.
2) The host trial questionnaire includes 91 individual items, with 64 of these items forming 6 scales (e.g. Perceived stress scale).
Reviewer Comment 2. The paper is missing a clear definition of the outcomes it will measure to assess the effect of the SWAT intervention. In particular what is the primary outcome and when will it be measured? We began to speculate as to whether the measurement was only at baseline, which didn’t seem to make sense. What we’d like to see is a clear indication of when the outcome assessment for the SWAT will be done.
Author Response 2. A clear explanation of the SWAT outcomes is now included in the manuscript as outlined below.
Outcome measurement
The primary outcome of the interest is the proportion of questionnaire completion. Individual item response rates will also be assessed in terms of completion of questions on infant feeding outcomes, and other outcomes such as healthcare utilization and parent well-being. This is because the SWAT Intervention text specifically refers to infant feeding, and so this study will examine whether the intervention influenced completion of these outcomes specifically.
A secondary outcome of interest is participant attitudes about questionnaire completion. Data on participant attitudes will be collected via questions included at the end of the CHErIsH questionnaire; the CHErIsH questionnaire can be completed online, in-person, or by phone based on participant preference, and so the SWAT questions can be similarly completed. Quantitative data for participant attitudes will be collected for all participants using the following two questionnaire items, which are rated from on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree’:
1. The infant-feeding related questions in the questionnaire were useful for gaining insight into how you feed your child.
2. The infant-feeding related questions in the questionnaire were appropriate for gaining insight into how you feed your child
Participants in the SWAT Intervention group will also be asked the following two questions, the first of which is closed-ended and rated from on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree’. The second SWAT Intervention question is a single open-ended question that allows participants to describe in their own words how the information on COS influenced their completion of the questionnaire. Both questions are as follows:
3. The information provided about Core Outcome Sets (COSs) influenced my completion of the questionnaire.
4. How did the information about Core Outcome Sets (COSs) influence your completion of the questionnaire?
Reviewer comment 3. We didn’t find Figure 1 helpful. It suggests that both groups are receiving the SWAT intervention but the text in the manuscript says ‘Information about the COS will be provided to a random sample of half of all participants in a brief paragraph..’, which suggests that not everyone gets it. The diagram would be better as a more standard CONSORT-esque figure with the timings as you work you way down the figure linked to the timing of outcome assessment.
Author Response 3. Figure 1 has been revised, as below. The SWAT Intervention group receive the COS information prior to questionnaire completion. The SWAT comparator receive this information once they have completed all questions, to provide a ‘debrief’ and ensure equal information provision for all participants. The SWAT is completed at baseline of the host trial only to determine effects on questionnaire completion and attitudes at time of information presentation, rather than effects over time. The decision to examine the effects of the COS information at baseline only is also guided by resource and time constraints.
Reviewer comment 4. The 4-item ‘SWAT questionnaire’ confused us a bit because not all questions are asked of all participants. Only questions 3 and 4 are unique to the SWAT intervention, we’re guessing that questions 1 and 2 are not specifically linked to the COS text that forms the SWAT intervention. Why not have questions 1 and 2 (common to both arms) as part of the main questionnaire and the SWAT questionnaire is then just questions 3 and 4? We also weren’t sure how the items on the questionnaire related to the outcomes of the SWAT. They are not response rates so we guess that they will be linked to attitudes. It would be good to know how this work will be done with the 5-point Likert scales and how the free-text in question 4 will be handled. Author response 4. The presentation of the ‘swat questionnaire’ has been revised, such that it is made clearer that two questions are asked of all participants, while two questions are only asked of the SWAT Intervention group. Questions 1 and 2 are directly related to the SWAT intervention text (which has now been included in the manuscript for clarity) as they relate directly to the infant feeding outcomes. These questions have also now been presented more clearly as addressing a secondary aim of the SWAT, which is to assess participant attitudes of questionnaire completion. Further information on how data will be analysed is now also included. The revised text is as follows:
Outcome measurement
The primary outcome of interest is the proportion of questionnaire completion. Individual item response rates will also be assessed in terms of completion of questions on infant feeding outcomes, and other outcomes such as healthcare utilization and parent well-being. This is because the SWAT Intervention text specifically refers to infant feeding, and so this study will examine whether the intervention influenced completion of these outcomes specifically.
A secondary outcome of interest is participant attitudes about questionnaire completion. Data on participant attitudes will be collected via questions included at the end of the CHErIsH questionnaire; the CHErIsH questionnaire can be completed online, in-person, or by phone based on participant preference, and so the SWAT questions can be similarly completed. Quantitative data for participant attitudes will be collected for all participants using the following two questionnaire items, which are rated from on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree’:
1. The infant-feeding related questions in the questionnaire were useful for gaining insight into how you feed your child.
2. The infant-feeding related questions in the questionnaire were appropriate for gaining insight into how you feed your child
Participants in the SWAT Intervention group will also be asked the following two questions, the first of which is closed-ended and rated from on a 5-point Likert scale from ‘strongly disagree’ to ‘strongly agree’. The second SWAT Intervention question is a single open-ended question that allows participants to describe in their own words how the information on COS influenced their completion of the questionnaire. Both questions are as follows:
3. The information provided about Core Outcome Sets (COSs) influenced my completion of the questionnaire.
4. How did the information about Core Outcome Sets (COSs) influence your completion of the questionnaire?
Analysis
Quantitative analysis: All questionnaire data will be entered into SPSS Version 24 software. Questionnaire response rates will be calculated for each of the SWAT Intervention and SWAT Comparator including proportion of completion of the CHErIsH questionnaire and individual item response rates. Chi squared tests will compare the proportion of the questionnaire completed for the two conditions. Potential differences between participant baseline characteristics (age, sex, education) will also be examined and should differences be observed, these will be controlled for.
Data from the SWAT Intervention group in response to the question 3 will be descriptively summarised in terms of participants’ mean attitude rating, standard deviation and range of ratings. As this data is only collected from the SWAT Intervention, inferential statistics will not be conducted.
Qualitative analysis: Responses to the open-ended question will be entered into NVivo 12 for qualitative data management and will be analysed using thematic analysis following Braun and Clarke (2006) guidelines. This will involve an iterative process of reading and re-reading the data, developing initial line codes, followed by categorisation and development of themes. However, if there is insufficient detail in the open-ended responses then they will be examined narratively.
Reviewer comment 5. Where is the SWAT outcome assessment done? Is it a postal questionnaire, linked to a visit for the host trial, or something else?
Author Response 5. The four SWAT questions are included at the end of the host trial questionnaire. As participants in the host trial can complete the questionnaire online, in-person or by phone, the SWAT questions can also be completed in this way. The following has been added to the text to clarify: the CHErIsH questionnaire can be completed online, in-person, or by phone based on participant preference, and so the SWAT questions can be similarly completed.
Reviewer 6. We weren’t clear why the researchers were measuring response rates for full questionnaire and individual items and which of these is considered the most important? And we assumed (but weren’t sure) that the completion of the host trial’s primary outcome was the trial outcome that you were concentrating on with regard to increasing response rates. Or is it all of the host trial outcomes?
Author response 6. This SWAT is measuring overall proportion of response rates and response rates for individual items, with both considered equally important in the context of this study. This is because core outcome sets (COS) are not typically the only outcomes to be measured in a trial, though they are the recommended minimum, so it is important to distinguish if the effects of COS provision extends to all outcomes, or just those directly related to the COS (for instance in this case, the infant feeding outcomes).
Reviewer comment 7. It would be useful to see a statement about how many people you think will be involved in the SWAT. This will be limited by the size of the CHErIsH trial so we’re not asking for a sample size calculation, just an indication of how many people are likely to be involved.
Author Response 7. The following text has been added to the paper: On average, 450 infants per annum are born to parents attending the primary care centre and during the 3 month recruitment period, It is anticipated that approximately 112 of these will be eligible for recruitment.
Reviewer comment 8. Can you say a bit more about why (and what) you will adjust for when looking at baseline imbalance. Any differences will be due to chance if the randomisation works though if the sample is small, it is true that these may lead to under or over-estimates of effect if the differences are things that affect your outcomes.
Author response 8. Participant age, sex, education will be controlled for as these have an impact on the study outcomes and therefore may impact on questionnaire completion rates. Adjusting for these variables will also be useful, as the reviewers say, because the potentially small sample size may lead to inaccurate estimation of effects.
Discussion
Reviewer comment 1. The SWAT does involve extra data collection (not all SWATs do) and we could speculate that this could reduce response rates to the host trial questionnaire. Could the authors comment on this?
Author Response 1. Yes, this SWAT includes three additional closed-ended questions (only 2 of which are asked in the SWAT Comparator group) and one open-ended question. The following has been added to the discussion to highlight that this is a potential weakness but that care has been taken to minimise the effects of this:
Inclusion of additional SWAT questions at the end of the CHErIsH trial questionnaire may also increase participant burden, which could impact on questionnaire completion rates. However, care was taken by the research team to develop questions that are as brief as possible to minimise this, and these questions are presented at end of CHErIsH questionnaire such that they their presence will potentially have minimal impact on completion.