https://orcid.org/0000-0002-7496-6727
Figures
Abstract
Introduction
REsilience and Activities for every DaY (READY) is an Acceptance and Commitment Therapy-based group resilience-training program that has preliminary empirical support in promoting quality of life and other psychosocial outcomes in people with multiple sclerosis (PwMS). Consistent with the Medical Research Council framework for developing and evaluating complex interventions, we conducted a pilot randomized controlled trial (RCT), followed by a phase III RCT. The present paper describes the phase III RCT protocol.
Methods and analysis
This is a multi-centre cluster RCT comparing READY with a group relaxation program (1:1 ratio) in 240 PwMS from eight centres in Italy (trial registration: isrctn.org Identifier: ISRCTN67194859). Both interventions are composed of 7 weekly sessions plus a booster session five weeks later. Resilience (primary outcome), mood, health-related quality of life, well-being and psychological flexibility will be assessed at baseline, after the booster session, and at three and six month follow-ups. If face-to-face group meetings are interrupted because of COVID-19 related-issues, participants will be invited to complete their intervention via teleconferencing. Relevant COVID-19 information will be collected and the COVID-19 Peritraumatic Distress scale will be administered (ancillary study) at baseline and 3-month follow-up. Analysis will be by intention-to-treat to show superiority of READY over relaxation. Longitudinal changes will be compared between the two arms using repeated-measures, hierarchical generalized linear mixed models.
Conclusion
It is expected that his study will contribute to the body of evidence on the efficacy and effectiveness of READY by comparing it with an active group intervention in frontline MS rehabilitation and clinical settings. Results will be disseminated in peer-reviewed journals and at other relevant conferences.
Citation: Giovannetti AM, Pakenham KI, Presti G, Quartuccio ME, Confalonieri P, Bergamaschi R, et al. (2022) A group resilience training program for people with multiple sclerosis: Study protocol of a multi-centre cluster-randomized controlled trial (multi-READY for MS). PLoS ONE 17(5): e0267245. https://doi.org/10.1371/journal.pone.0267245
Editor: Carla Pegoraro, PLOS, UNITED KINGDOM
Received: April 9, 2021; Accepted: April 1, 2022; Published: May 2, 2022
Copyright: © 2022 Giovannetti et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The project is supported by FISM - Fondazione Italiana Sclerosi Multipla – cod. 2016/B/3 and financed or co-financed with the ‘5 per mille’ public funding. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: AMG reports a grant from Rehabilitation in Multiple Sclersis (RIMS) during the conduct of the study. AS reports grants from Fondazione Italiana Sclerosi Multipla (FISM), during the conduct of the study; personal fees from Biogen Idec, Merck Serono, Novartis, Almirall, and Excemed. As a staff member of the University of Queensland and co-Author of the READY program, KP receives royalties from UniQuest for commercial (not research) licensing arrangements entered into by third parties who want to deliver the program. PC has received honoraria for speaking or consultation fees from Novartis and Biogen, has received funding for travel to attend scientific events or speaker honoraria from Merck Serono, Biogen Idec, Teva and Roche. He has also received institutional research support from Merk-Serono, Novartis and Roche. He is also principal investigator in clinical trials for Biogen, Merck Serono, and Roche. This does not alter our adherence to PLOS ONE policies on sharing data. All the other authors reports no competing interests.
Introduction
Multiple sclerosis (MS) is a chronic disease that affects the central nervous system through processes of demyelination and degeneration that ultimately cause neuronal damage and axonal loss [1]. The prevalence of MS is rising globally [2]. The most recent study on the global prevalence of MS estimated that 2.8 million people worldwide have MS [3]. Italy, with more than 125,500 persons with MS (PwMS), has one of the highest prevalence rates [4]. Because MS onset is usually between 20 and 40 years old [5], the disease has the potential to severely impact most life domains [6, 7]. Moreover, PwMS often have to cope with uncertainty about disease progression, loss of function, changes in life roles and a variety of symptoms [8]. These stressors can evoke a deep sense of personal vulnerability [9] and thwart the illness adjustment process [10]. Indeed, evidence shows that PwMS have poorer quality of life (QoL) than healthy controls and people with other chronic diseases [11, 12]; with a recent meta-analysis reporting a lifetime prevalence of 30.5% for depression and 22.1% for anxiety symptoms in this population [13]. In addition, research evidence supports the association between psychological stress and subsequent relapses in MS, with the occurrence of stressful life events purported to lead to a greater risk of relapse [14].
Evidence suggests that resilience plays a key role in alleviating the adverse effects of stress and sustaining mental health in adversity [15]. It entails the process of negotiating, managing and adapting to significant stressors or trauma through drawing on internal (i.e. mindfulness, acceptance, cognitive flexibility and active coping), and external (i.e. social support, financial capital and community services) resources [16]. When facing adversity and stressful situations, people with lower resilience have a higher risk of experiencing poorer QoL, and greater distress and relational difficulties [17], and adopting unhealthy behaviors, which in turn are likely to negatively affect physical health [18]. As reported by Strike and Steptoe, poor psychosocial functioning and exposure to prolonged stress are likely to adversely impact physical health through physiological stress reactions such as hypertension, blood pressure increases, pro-inflammatory cytokines and the development of metabolic syndrome [19]. Given the evidence showing PwMS have lower resilience than the general population and people with other chronic diseases [20], they are particularly vulnerable to the adverse effects of stress. Therefore, evidence-based interventions aimed at fostering resilience are pivotal in helping them positively cope with their illness-related stressors and promote well-being and QoL.
Resilience-training interventions have been shown to promote a range of positive psychosocial outcomes in people with chronic illnesses [21], including QoL, anxiety, depression, perceived stress and well-being in adults with cancer [22–24], congenital heart disease [25], diabetes [26], neurofibromatosis [27], and MS [28]. However, the quality of many of these studies is sub-optimal [29], and Chmitorz et al. identified three main methodological problems: definitions of resilience as trait or a composite of resilience factors rather than as a process; the use of psychometrically weak instruments; the omission of one or more key study design elements (e.g., a priori sample size calculation, adequate comparator, sufficient baseline diagnostics, long-term follow-up assessment, adverse effects assessments, participant satisfaction evaluation, and multi-centre study designs) [30].
In the last decade, an Australian team has developed and tested an Acceptance and Commitment Therapy (ACT)-based group resilience-training program called REsilience and Activities for every DaY (READY). The READY program was initially applied in a workplace setting [31, 32], and then adapted and successfully implemented with different health conditions: cancer [33, 34], diabetes [35], and MS [28]. The Australian case series study on READY for MS showed that the program had beneficial impacts on resilience, QoL, depression, stress and protective factors (managing difficult thoughts, values and acceptance) in PwMS [28].
READY is informed by ACT, which is a third wave cognitive behavior therapy that aims to promote psychological flexibility. According to ACT, psychological flexibility involves behaving consistently with personal values even in the presence of psychological discomfort [36]. Psychological flexibility is established through the following six processes: (1) acceptance–openness to experience, (2) cognitive defusion–observing thoughts rather than taking them literally, (3) present moment awareness–mindfulness, (4) self-as-context–contact with a sense of self that is continuous and provides flexible perspective taking, (5) values–freely chosen personally meaningful life directions, (6) committed action–values-guided effective action [36]. The READY program uses these six core ACT processes to target five empirically supported resilience protective factors: cognitive flexibility (defusion), acceptance, meaning, social connectedness and values-based action.
ACT is as an empirically supported intervention for the promotion of mental health across a wide range of contexts [37–40], and in people with various health conditions [33–36, 41–47]. A recent review of meta-analyses demonstrated that ACT is superior to inactive controls (e.g. waitlist, placebo), treatment as usual, and most active intervention conditions (with the exception of other cognitive behavioral therapy-based intervention) [48].
Based on the empirical support for ACT and the READY program, we developed a research protocol to evaluate the efficacy of READY for MS (here-on called READY) in Italy. We designed the project following the Medical Research Council (MRC) framework for developing and evaluating complex interventions, which entails a multi-phased approach involving a pre-clinical research phase and a final phase in which the intervention is introduced into the health service [49]. Hence, the project consisted of two phases: 1) a pilot randomized controlled trial (RCT) with a nested qualitative study [50]; 2) a multi-centre phase III RCT, which is the focus of the present study. The second phase had an ancillary study that evaluated the impact on psychologists of their training in READY [51] and the effectiveness of READY delivered via frontline Italian health services by psychologists trained in the intervention [52].
Results from the pilot study showed that although the READY program was well accepted by PwMS and suitable for Italian clinical settings, there were no between-arm differences in any patient-reported outcomes. Three methodological limitations may explain the absence of statistical superiority of READY over relaxation: small sample size, short follow-up, and a ceiling effect with the primary outcome (QoL). Qualitative data showed that participants viewed READY as superior to relaxation; a finding that converged with four non-significant statistical trends (resilience, psychological flexibility, acceptance and defusion) supporting READY efficacy. Consistent with the ACT psychological flexibility framework, participants viewed their improvements in resilience and health-related QoL as being due to the acquisition of skills related to the six core ACT processes [50].
The trial steering committee (TSC) and an international expert panel discussed the pilot study findings in two dedicated meetings (January 2020). The panel discussion was structured on the “PICO” (Population, Intervention, Comparator, Outcomes) format [53]. Panel recommendations are reported in Box 1.
Box 1. Panel recommendations
- The READY and control interventions should be delivered by different facilitators.
- Control intervention facilitators should not have had prior training in mindfulness or ACT nor receive such training for the duration of the study.
- A six-month post-intervention follow-up should be scheduled in addition to the 3-month follow-up (primary endpoint).
- Following Chmitorz et al.’s suggestion that resilience should be the primary outcome measure in resilience interventions, and based on the trend differences observed in the pilot RCT, the Connor-Davidson Resilience Scale 25 (CD-RISC 25) was chosen as the primary outcome.
Materials and methods
This is a multi-centre cluster RCT assessing the superiority of READY over relaxation, with three and six-month follow-ups. The study follows the CONSORT guidelines for RCTs on social and psychological interventions (CONSORT-SPI 2018) [54]. The study protocol was designed following the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidelines (S1 Appendix) [55], and The SPIRIT-PRO Extension [56]. The SPIRIT schedule of enrolment is reported in Fig 1, and the CONSORT flowchart in Fig 2.
Note. * CD-RISC 25 is administered after informed consent is signed.
We hypothesize that compared to the control arm (relaxation), READY participants would show greater improvements on the primary outcome of resilience and on the secondary outcomes of mood, health-related QoL, well-being and psychological flexibility. The primary end-point is the between-arm difference from baseline (T0) to three-month follow-up (T2) changes in resilience scores.
The protocol received ethical clearance from the ethics committee of the Fondazione IRCCS Istituto Neurologico Carlo Besta (15th April 2020, internal ref: 71; amendment approved 18th November 2020, internal ref: 78) and it has been evaluated by each participating centre’s ethics committee. After obtaining Ethics Committee approvals from all the participating centres, the study will be run at the MS/rehabilitation units of eight university hospitals, research hospitals, general hospitals or community health services across Italy.
Eligibility criteria
Participant inclusion criteria are: MS diagnosis [57]; age ≥18 years; written informed consent; CD-RISC 25 score < 83, which indicates that the person could still improve his/her level of resilience; able to attend group sessions, and fluent Italian speaker. PwMS will be excluded from the study if one or more of the following criteria are met: severe cognitive compromise (Mini Mental State Examination <19); psychosis or other serious psychiatric conditions; psychotherapy in the preceding six months; prior formal training in mindfulness methods or current meditation practice; severe suicidality, including ideation, plan and intent; one or more relapses in the previous month; corticosteroid treatment during the previous month; other serious medical conditions in addition to MS; current pregnancy; MS diagnosis for less than three months.
Recruitment and trial procedures
A flyer, which includes a general overview of the study and contact details, will be sent via e-mail to PwMS by the MS Centre team. People who show interest in participating in the study will be contacted by the study Principal Investigator (PI) or centre PI. Subsequently, one trained clinical psychologist (READY facilitator) will make an appointment with those patients who met the inclusion criteria and agreed to participate in the study, and check all eligibility criteria. She/he will send an e-mail to the participant with a link to the website containing the set of questionnaires. The assessment will last about 55 minutes at each time-point (additional 20 minutes at T1 and T2 for completing satisfaction questionnaires). Each centre will collect information on the number of people approached, screened, and eligible prior to random assignment, including reasons for non-enrolment. Participants will then be assigned to the READY or relaxation in a 1:1 ratio using the method of minimization (two factors: Centre and CDRISC score < 50 and ≥ 50) [58]. Treatment assignment will be provided by an independent randomization unit, using a computer-based algorithm [59]. Confirmation e-mails will be sent to the study PI. The interventions will start within two weeks of the baseline assessment.
Confidentiality
All study-related information will be stored securely at the study site. Data collection, process, and administrative forms will be identified by a coded ID number only to maintain participant confidentiality. All records that contain names or other personal identifiers, will be stored separately from study records identified by code number. All local databases will be secured with password-protected access systems. Forms, lists and any other listings that link participant ID numbers to other identifying information will be stored in a separate, locked file in an area with limited access. Audio recordings of the sessions will be deleted immediately after intervention fidelity assessments are completed (no later than one week after a session).
Pre-study interview and informed consent (baseline visit, T0)
During the pre-study evaluation each potential participant will receive comprehensive verbal and written information about the nature and purpose of the study. Written, signed informed consent will be obtained in accordance with the Declaration of Helsinki and the GCP Guidelines of the EU.
Assessments
Patient reported outcome measures (PROMs) will be administered at T0, after the booster session (T1, 12 weeks after baseline visit), at three (T2), and six month follow-ups (T3). Participants will complete the purpose-built Participant Satisfaction Questionnaire at T1 and T2. They will receive an email with the link to the website containing the set of questionnaires. Additional process data will capture participant attendance and homework completion, and facilitator perspectives on a weekly basis. At T0 the patient’s referring neurologist will report the following clinical information on the case report form: age at MS diagnosis; expanded disability status sale (EDSS) score [60]; MS course (relapsing remitting, primary progressive, secondary progressive); and ongoing disease modifying treatment. Neurologists will update the occurrence of new relapses at each time-point.
Interventions
Each intervention group will have 8–10 participants. A total of 24 groups will be run (12 READY and 12 relaxation). The READY facilitators are member of “The psychologist network of the Italian Multiple Sclerosis Association (AISM)” who successfully completed “The ACT and be READY for MS Training Program” [51]. The relaxation facilitators are psychologists not involved in “The ACT and be READY for MS Training Program” and with no prior training in ACT or mindfulness interventions. Each group will be run in a dedicated room of the corresponding centre. Participants and facilitators will be asked not to disseminate their intervention information or materials. Each centre will organize READY and relaxation sessions at different times/days in order to prevent possible contact between READY and relaxation participants.
READY.
READY is an adult ACT informed group resilience-training program, in which resilience is metaphorically described as a shield composed of five life domains (thinking, feeling, doing, relations and being). Within each life domain an empirically supported key resilience protective factor is highlighted and targeted by the intervention. These protective factors reflect one or more of the core ACT processes. Although for ease of presentation and understanding, the shield contains seemingly separate facets, the domains and protective factors are dynamic and overlapping. The ACT processes impact multiple domains and protective factors. In particular, mindfulness and self-as-context occupy a central pivoting role due to their diffuse and synergistic psychosocial effects (Fig 3) [61]. Content of the seven weekly sessions is as follows: an introductory module (Introduction to the READY Resilience Model), five modules focusing on the six ACT processes (Mindfulness, Acceptance, Cognitive Defusion, Self-as-context, Values and Meaningful Action), and a review module (Review and Future Planning). The booster session provides a review of the program content (S2 Appendix). The program has a facilitator manual, participant workbook, and audio recordings of mindfulness exercises. Throughout the program, participants are encouraged to share their progress and experience of applying the READY strategies and techniques. It incorporates a blend of psychoeducation and experiential exercises, combined with readings and homework exercises that participants practice between sessions [28, 50].
Note. Life domains are in bold, ACT processes reflecting resilience protective factors (italics) are reported in the relevant life domain (shield clove) or in the centre of the shield if they have a central pivoting role.
Relaxation.
The control intervention consists of a group relaxation program based on autogenic training [62]. The program matches the study intervention in number of sessions (seven) and schedule (S3 Appendix) but not in session content and length (1-hour). The program has a facilitator manual, participant workbook, and audio recordings of relaxation exercises.
Intervention fidelity and supervision.
Each session (READY and relaxation) will be audio-recorded and the file sent via e-mail to the study PI immediately after recording. The procedure for monitoring intervention fidelity includes the following actions.
After each session, facilitators complete a purpose-build Session Fidelity Checklist and The Acceptance and Commitment Therapy Fidelity Measure (ACT-FM; only READY facilitators) [63], and send them to the study PI. The Session Fidelity Checklist lists the sequence of components for each session as reported in the Facilitator Manual. It also includes a section for clinical notes. Two versions are available, one for READY (S4 Appendix), and one for the relaxation program (S5 Appendix).
The 25-item ACT-FM evaluates adherence to the ACT therapeutic style. Items are rated on a 4-point Likert scale. It yields two scores that reflect the overarching dimensions of ACT consistent and inconsistent therapist behaviour. Each of these dimensions has four subscales: 1) ACT Consistent Therapist Stance; 2) ACT Inconsistent Therapist Stance; 3) ACT Consistent Open Response Style; 4) ACT Inconsistent Open Response Style; 5) ACT Consistent Aware Response Style; 6) ACT Inconsistent Aware Response Style; 7) ACT Consistent Engaged Response Style; 8) ACT Inconsistent Engaged Response Style [63].
After each session the study PI will review these materials to check for self-reported discrepancies with the manual and/or inconsistencies with ACT principles (only READY facilitators). In the case of self-reported discrepancy (or inconsistency with ACT), the study PI will contact the facilitator and discuss the session. The study PI will be available for supervision anytime.
Two audio-recordings randomly selected for each facilitator (one from session 2 or 3 and one from sessions 4 to 7) will be assessed by the study PI using the Session Fidelity Checklist and ACT-FM (only for READY) [63]. READY recordings will also be independently assessed by another ACT expert (TSC member), and in the case of a discrepancy in assessors’ ratings, they will discuss the relevant data until a consensus is reached. If low intervention fidelity is identified, the study PI will contact the facilitator to critically discuss the session and the quality of her/his facilitation and the facilitator’s next session will also be assessed.
Low fidelity will be determined if at least one of the following criteria are satisfied: A discrepancy with the content manual in more than one section of the Session Fidelity Checklist; For READY facilitators an ACT-FM score < 5 for the Stance Consistent section (items 1–4) and a score < 11 for the other sections combined (these criteria were defined after discussion with the ACT-FM Authors).
Patient and public involvement statement
PwMS and an AISM member were involved at several stages of developing the multi-phased project. We received input on the READY and relaxation interventions from participants in the pilot study via both questionnaires and personal interviews [50]. The results of the pilot study were discussed in a dedicated meeting involving the TSC and an expert panel and were used to design the present RCT. The AISM Director of Health Care Professional and Client Services unit had been a member of the TSC until June 2020. An expert MS patient is now a member of the TSC and a co-author of the present paper. We will disseminate key study findings to PwMS with assistance from AISM.
Outcome measures
Primary outcome measure.
The Connor-Davidson Resilience Scale 25 (CD-RISC 25) is used to assess psychological resilience. It is composed of 25 items, each rated on a 5-point scale (0–4), with higher scores reflecting greater resilience. The scale has demonstrated good psychometric properties (i.e. its internal consistency, test–retest reliability, and convergent and divergent validity) [64]. The CD-RISC scores have been shown to increase with treatments hypothesized to enhance resilience [65]. It also obtained the highest rating in a methodological review of resilience measures [16].
Secondary outcome measures.
The following PROMs will be administered in the order they are presented.
Anxiety and depression. The Hospital Anxiety and Depression Scale (HADS) is a well-validated measure that consists of two seven-item subscales to assess anxiety and depressive symptoms. Higher scores indicate higher levels of depressive and anxiety symptoms [66]. Unlike many similar measures, the HADS excludes somatic symptoms of anxiety and depression, which may overlap with physical illness symptoms [66].
Affect. The Positive and Negative Affect Schedule (PANAS) consists of two 10-item mood scales and it is a self-report measure of positive and negative affect. Respondents rate the extent to which they have experienced each particular emotion during the last two weeks on a 5-point Likert scale. The PANAS has been shown to be a reliable and valid measure of affect [67].
MS-related QoL. The Multiple Sclerosis Quality of Life-54 (MSQOL-54) is a MS-specific measure of Health-related QoL. It comprises the generic 36-item Short-Form (SF-36), plus 18 MS-specific items [68]. The 54 items are organized into 12 multi-item and two single item subscales. As for the SF-36, two composite scores (Physical and Mental Health Composite) are derived by combining scores of the relevant subscales. The MSQOL-54 has well documented content, construct and discriminative validity and reliability [68, 69].
Health-related QoL. The EQ-5D-3L is a preference-based health-related QoL measure with one question for each of the five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. It also includes a Visual Analog Scale for perceived health status ranging from 0 (the worst possible health status) to 100 (the best possible health status) [70].
The 14-item Mental Health Continuum Short Form (MHC-SF) measures social, emotional and psychological well-being. Respondents rate the frequency of various experiences in the past month on a 6-point Likert scale. The MHC-SF has shown good psychometric properties [71].
Psychological flexibility. The Multidimensional Psychological Flexibility Inventory (MPFI) is a 60-item self-reported questionnaire assessing psychological flexibility and inflexibility. Previous studies confirmed the two-factor second order model in both the original English MPFI and the Italian MPFI version [72, 73]. To limit participants’ burden in questionnaires completion, only the psychological flexibility subscale (30 item) will be included in the present study. The psychological flexibility subscale includes 6 dimensions of the Hexaflex model (i.e. acceptance, mindfulness, self-as-context defusion, contact with values, and committed action) [72]. Each dimension is evaluated via 5 items. Items are rated on a 6-point Likert scale, from 1 “never true” to 6 “always true”. Higher scores indicated higher level of psychological flexibility. The questionnaire in general and its subscales have demonstrated good psychometric properties in both clinical and nonclinical samples [40, 74–76]. Moreover, findings showed that MPFI subscales have good validity in terms of responsiveness to change over time [72].
Satisfaction with the intervention. Four purpose-built questionnaires (two for each intervention) explore satisfaction with READY or relaxation at post-intervention and 3-month follow-up.
Adherence to the intervention. After each session facilitators will collect information on participant’s attendance and ask the participants to rate their level of commitment to homework activities.
We will use the Italian versions of the HADS [77], PANAS [78], MSQOL-54 [79], EQ-5D-3L [80], MHC-SF [81], and MPFI [73]. For the CD-RISC 25 we will use the unpublished Italian version [courtesy of Davidson].
Data analysis
Sample size calculation.
The sample size calculation was based on the pilot study results [50], which showed a mean change in the CD-RISC 25 at 3-month follow-up in the READY arm equal to 15.61 (SD 13.63, n = 18) vs. a mean change in the relaxation arm equal to 5.95 (SD 14.11, n = 19). The intra-cluster correlation coefficient (ICC) was equal to 0.08.
A sample size of 12 clusters per study arm (total number of clusters = 24) with 10 PwMS per cluster (total sample size 240) achieves 94.6% power to detect a mean difference of 9.66 between the two arms [82]. We made the following assumptions: SD of change in the CD-RISC 25 at 3-month of 14.53; ICC of 0.10; and alpha value of 0.05.
Based on these figures (i.e., total sample size 240, 24 clusters, 10 subjects per cluster), we computed the study power considering different drop-out scenarios (S6 Appendix).
Statistics.
Analyses will be carried out in accordance with the pre-specified statistical analysis plan and performed by study personnel blind to the participants’ assignment and to those providing the interventions.
Baseline variables’ standardized mean differences (for clustered data) between arms will be computed to measure potential unbalance.
Longitudinal changes will be analyzed using repeated measures hierarchical (patients nested in clusters) generalized linear mixed models, accounting for the cluster effect (using random intercepts for clusters) and adjusted for those baseline covariates resulting unbalanced between arms. As reported above, the between-arm difference from baseline (T0) to three-month follow-up (T2) changes in resilience scores is the primary endpoint. To mitigate the risk of inflating Type I error given the small number of clusters, we will use the Kenward-Roger degrees of freedom correction, which does not rely on the assumption of fixed cluster sizes [83].
All group comparisons will be carried out according to the intention-to-treat principle. That is, participants will be analysed in the arm (READY or relaxation) to which they were assigned. We will use multiple imputations for missing data where appropriate (i.e. drop-out, missing item/questionnaire) [84]. In addition, we will carry out a per-protocol analysis and assess the sensitivity of the results to excluding patients who missed three or more READY sessions.
The study will be overseen by an independent data and safety monitoring committee (DSMC) consisting of three members with expertise in biostatistics, complex interventions, and psychology. No interim analyses will be conducted. The DSMC will monitor accrual and retention of participants. Protocol amendments are made in consultation with the DSMC.
Strategies for limiting COVID-19 impacts on the study
Conducting a study that involves face-to-face group interventions during the COVID-19 pandemic exposes it to the risk of having to abort interventions due to the emergence of local COVID-19 infection ‘hot-spots’. The TSC carefully considered three options. First, pausing the study until the pandemic situation is under control. Due to the uncertainties of predictions about when the pandemic will be more manageable, this option was excluded. Second, delivering the interventions online. This option was also excluded because it would violate the MRC framework guidelines by nullifying the pilot RCT which, as purposed, provided data that informed the design of the present multi-centre RCT. Third, retaining the group delivery of both interventions with an option for teleconference delivery in the case of COVID-19 infection threats. This option was accepted by the TSC and involved a set of actions, summarized in Box 2.
Box 2. TSC action plan for limiting COVID-19 impacts on the study
- The study PI will have weekly contact with the participating centres in order to monitor changes in the local spread of COVID-19 and to take prompt infection control actions as necessary in accordance with guidelines from the Italian Government and local health authorities.
- In the case of one or more participating centres not being able to start the study up to four months after the commencement of enrolling participants in other centres, the active centres will run the remaining groups in their place (competitive enrolment).
- If face-to-face group meetings are interrupted because of COVID-19 related-issues (e.g., local lockdown), all affected participants will be invited to complete their intervention via teleconferencing and the assessments as scheduled.
- Relevant COVID-19 information will be collected (i.e., COVID-19 infected; family member infected; COVID-19 impacts on employment; risk perceptions of COVID-19 infection because of MS; perceived additional burden of MS treatment/rehabilitation because of COVID-19).
- During data analysis a sensitivity analysis will be undertaken to explore any effects of the teleconference intervention delivery relative to the face-to-face delivery.
The study results will be published in peer-reviewed journals, presented at conferences and a lay summary will be sent to participants. The TSC will suggest topics for presentation or publication and will circulate them to the PI of each participating centre. Topics suggested by a member of the participating centre should be approved by the TSC and the person making the suggestion may be considered as the lead author.
The study was registered on the ISRCTN registry (isrctn.org Identifier: ISRCTN67194859) the 14th May 2020.
Discussion
The primary purpose of this trial is to assess the effects of a brief, structured group resilience-training intervention on resilience in PwMS over the nine-month study period. A secondary purpose is to assess the effects of the intervention on participant’s mood, QoL, well-being and psychological flexibility. Considering that the intervention was developed in Australia and only recently applied in Italy with preliminary empirical support [50], it is important to further investigate the efficacy of READY in the Italian context. Recent literature reviews on resilience have emphasized the need for more methodologically rigorous research on the effects of resilience-training interventions [29, 30]. The design of the present study not only overcomes all the limitations reported in these reviews, it adheres to the MRC framework for developing and evaluating complex interventions [49], and it also aligns with all the recommendations for future research identified by Öst in his systematic review and meta-analysis on the efficacy of ACT interventions with the exception of a follow-up of at least one year [85].
Within this methodological context, this study will contribute to the body of evidence on the efficacy and effectiveness of READY by comparing it with an active group intervention in frontline MS rehabilitation and clinical settings. It is expected that READY will cultivate targeted resilience protective factors that will help PwMS effectively manage MS-related stressors. Moreover, the READY program is relatively brief and highly structured, two characteristics that increase its affordability and ease of dissemination.
It should also be noted that the psychological flexibility processes that underpin the READY intervention have been shown to protect people from the adverse mental health impacts of COVID-19 [86, 87]. For this reason, the Italian version of the COVID-19 Peritraumatic Distress scale will be administered (and completed on a voluntary basis) at baseline and 3-month follow-up (primary endpoint) as part of an ancillary study [86, 88].
Limitations and measures to minimize bias
Two study limitations are noted. Blinding of patients is not possible due to the type of the study interventions. The READY and relaxation interventions are similar in frequency and number of sessions, but not in duration. Despite these limitations the following measures will be used to increase methodological rigor. The statistical analyses will be performed by study personnel blind to the participants’ randomization to the two intervention conditions and to those providing the interventions. Electronic versions of the study PROMs will be used to ensure the data entered is of high quality. The TSC will monitor adherence to the study protocol and overall study quality. Finally, an independent DSMC will oversee the study procedures, recruitment, and data flow. The study PI and another ACT expert will monitor the intervention fidelity audio recordings and ratings of READY and relaxation sessions. Facilitators will only receive detailed information on the group intervention (READY or relaxation) they conduct. For this reason, facilitator meetings will be run separately for READY and relaxation facilitators. Members of the Clinical Psychology Expert Panel will also be available to discuss any issues facilitators have in relation to delivering the interventions.
Supporting information
S4 Appendix. READY for MS session fidelity checklist.
https://doi.org/10.1371/journal.pone.0267245.s004
(DOCX)
S5 Appendix. Relaxation session fidelity checklist.
https://doi.org/10.1371/journal.pone.0267245.s005
(DOCX)
Acknowledgments
Author want to thank Prof. Marta Bassi, Dr. Andrea Giordano, Prof. Stefan Gold, Prof. Christopher Heesen and Dr. Jana Pöttgen for their precious feedback on the study protocol.
Collaborators
Multi_READY for MS Trial Steering Committee: AMG, AS, KIP, GP, PK. Independent Data and Safety Monitoring Committee: S Gold, M Bassi, MP Sormani. Data Management and Analysis Committee: M Copetti, AG, AMG, AS. Clinical Psychology Expert Panel: AMG, KIP, GP and J Pöttgen. Centres and investigators: Fondazione IRCCS Istituto Neurologico Carlo Besta, Unit of Neuroepidemiology: AG, AMG, AS. Fondazione IRCCS Istituto Neurologico Carlo Besta, Unit of Neuroimmunology and Neuromuscular Diseases, Multiple Sclerosis Centre: PC, Rui Quintas, Milda Černiauskaitė. San Camillo-Forlanini Hospital, Roma: Carla Tortorella, MEQ. AISM Rehabilitation Service of Genoa, Italian Multiple Sclerosis Society, Genova: GB, Miranda Giuntoli, Annalisa Garaventa. Neurology Clinic, Multiple Sclerosis Centre, University Hospital Policlinico Vittorio Emanuele, Catania, Italy: FP, Eleonora Chisari, Chiara Vona. Laboratorio di neuropsicologia, UOSD psicologia clinica e UOC neurologia, ASST Lariana: MG, Samuela Turati. Centro Sclerosi Multipla, Divisione di Neurologia Generale, IRCCS Fondazione Istituto Neurologico Nazionale C. Mondino di Pavia: RB, Ambrogia Ornella Riolo, Marta Picascia. Dipartimento Riabilitazione ASLUmbria2: MM, Serena De Bigontina. Centro Malattie Demielinizzanti e Laboratori di Neurologia Sperimentale, Clinica Neurologica, Università di Perugia: MDF, Giuliana Costantini, Luciana Ciaccassassi.
References
- 1. Compston A, Coles A. Multiple sclerosis. Lancet. 2002;359: 1221–1231. pmid:11955556
- 2. Leray E, Moreau T, Fromont A, Edan G. Epidemiology of multiple sclerosis. Rev Neurol (Paris). 2016;172(1): 3–13. pmid:26718593
- 3. Walton C, King R, Rechtman L, Kaye W, Leray E, Marrie RA, et al. Rising prevalence of multiple sclerosis worldwide: Insights from the Atlas of MS, third edition. Mult Scler. 2020;26(14): 1816–1821. pmid:33174475
- 4.
Associazione Italiana Sclerosi Multipla Onlus. Barometro della Sclerosi Multipla 2020. [cited 1 April 2021]. Available from https://www.camera.it/application/xmanager/projects/leg18/attachments/upload_file_doc_acquisiti/pdfs/000/003/998/AISM_8.9.2020_.pdf
- 5.
Dupont S. Multiple sclerosis. In: Ayers S, Baum A, McManus C, Newman S, Wallston K, Weinman J, et al., editors. Cambridge handbook of psychology, health and medicine; London: Cambridge Univ Press; 1997. pp. 5385–5440.
- 6. Giordano A, Granella F, Lugaresi A, Martinelli V, Trojano M, Confalonieri P, et al. Anxiety and depression in multiple sclerosis patients around diagnosis. J Neurol Sci. 2011;307(1–2): 86–91. pmid:21621796
- 7. Schiavolin S, Leonardi M, Giovannetti AM, Antozzi C, Brambilla L, Confalonieri P, et al. Factors related to difficulties with employment in patients with multiple sclerosis: a review of 2002–2011 literature. Int J Rehabil Res. 2013;36: 105–111. pmid:23238667
- 8. Dennison L, Moss-Morris R, Chalder T. A review of psychological correlates of adjustment in patients with multiple sclerosis. Clin Psychol Rev. 2009;29(2): 141–53. pmid:19167801
- 9. Giovannetti AM, Brambilla L, Torri Clerici V, Antozzi C, Mantegazza R, Černiauskaitė M, et al. Difficulties in adjustment to multiple sclerosis: vulnerability and unpredictability of illness in the foreground. Disabil Rehabil. 2017;39(9): 897‐903. pmid:27206327
- 10. Aikens JE, Fischer JS, Namey M, Rudick RA. A replicated prospective investigation of life stress, coping, and depressive symptoms in multiple sclerosis. J Behav Med. 1997;20: 433–45. pmid:9415854
- 11. Siegert RJ, Abernethy DA. Depression in multiple sclerosis: a review. J Neurol Neurosurg Psychiatry. 2005;76: 469–75. pmid:15774430
- 12. Korostil M, Feinstein A. Anxiety disorders and their clinical correlates in multiple sclerosis patients. Mult Scler J. 2007;13: 67–72. pmid:17294613
- 13. Boeschoten RE, Braamse AMJ, Beekman ATF, Cuijpers P, van Oppen P, Dekker J, et al. Prevalence of depression and anxiety in Multiple Sclerosis: A systematic review and meta-analysis. J Neurol Sci. 2017;372: 331‐341. pmid:28017241
- 14. Mohr DC, Hart SL, Julian L, Cox D, Pelletier D. Association between stressful life events and exacerbation in multiple sclerosis: a meta-analysis. BMJ. 2004;328(7442): 731–735. pmid:15033880
- 15. Leppin AL, Bora PR, Tilburt JC, Gionfriddo MR, Zeballos-Palacios C, Dulohery MM, et al. The efficacy of resiliency training programs: a systematic review and meta analysis of randomized trials. PLoS One. 2014;9(10): e111420. pmid:25347713
- 16. Windle G, Bennett KM, Noyes J. A methodological review of resilience measurement scales. Health Qual Life Outcomes. 2011;9(1): 8. pmid:21294858
- 17. Min JA, Yoon S, Lee CU, Chae JH, Lee C, Song KY, et al. Psychological resilience contributes to low emotional distress in cancer patients. Support Care Cancer. 2013;21(9): 2469–76. pmid:23604453
- 18. Rozanski A, Blumenthal J, Kaplan J. Impact of psychosocial factors on the pathogenesis of cardiovascular disease and implications for therapy. Circulation. 1999;99: 2192–2217. pmid:10217662
- 19. Strike PC, Steptoe A. Psychosocial factors in the development of coronary artery disease. Prog Cardiovasc Dis. 2004;46: 337–47. pmid:14961456
- 20. Terrill AL, Molton IR, Ehde DM, Amtmann D, Bombardier CH, Smith AE, et al. Resilience, age, and perceived symptoms in persons with long-term physical disabilities. J Health Psychol. 2016;21: 640–649. pmid:27165965
- 21. Kim GM, Lim JY, Kim EJ, Park SM. Resilience of patients with chronic diseases: A systematic review. Health Soc Care Community. 2019;27(4): 797–807. pmid:30027595
- 22. Loprinzi CE, Prasad K, Schroeder DR, Sood A. (2011). Stress Management and Resilience Training (SMART) Program to decrease stress and enhance resilience among breast cancer survivors: A pilot randomized clinical trial. Clin Breast Cancer. 2011;11: 364–368. pmid:21831722
- 23. Ye ZJ, Liang MZ, Qiu HZ, Liu ML, Hu GY, Zhu YF, et al. Effect of a multidiscipline mentor-based program, Be Resilient to Breast Cancer (BRBC), on female breast cancer survivors in mainland China—A randomized, controlled, theoretically-derived intervention trial. Breast Cancer Res Treat. 2016;158: 509–522. pmid:27400910
- 24. Sansom-Daly UM, Wakefield CE, Bryant RA, Ellis S, Doolan E, Cohn RJ. Adapting evidence based psychological therapy to the computer screen for adolescent and young adult cancer survivors: Preliminary results from the ‘recapture life’ randomized controlled trial. Asia Pac J Clin Oncol. 2014;10: 36.
- 25. Kovacs AH, Bandyopadhyay M, Grace SL, Kentner AC, Nolan RP, Silversides CK, et al. Adult congenital heart disease coping and resilience (ACHD-CARE): Rationale and methodology of a pilot randomized controlled trial. Contemp Clin Trials. 2015;45: 385–393. pmid:26546067
- 26. Bradshaw BG, Richardson GE, Kumpfer K, Carlson J, Stanchfield J, Overall J, et al. Determining the efficacy of a resiliency training approach in adults with type 2 diabetes. Diabetes Educ. 2007;33: 650–659. pmid:17684166
- 27. Vranceanu AM, Riklin E, Merker VL, Macklin EA, Park ER, Plotkin SR. Mind-body therapy via videoconferencing in patients with neurofibromatosis. Neurology. 2016;87: 806–814. pmid:27449066
- 28. Pakenham KI, Mawdsley M, Brown FL, Burton NW. Pilot evaluation of a resilience training program for people with multiple sclerosis. Rehabil Psychol. 2018;63(1): 29–42. pmid:29154558
- 29. Forbes S, Fikretoglu D. Building resilience: The conceptual basis and research evidence for resilience training programs. Rev Gen Psychol. 2018;22(4): 452–468.
- 30. Chmitorz A, Kunzler A, Helmreich I, Tüscher O, Kalisch R, Kubiak T, et al. Intervention studies to foster resilience—A systematic review and proposal for a resilience framework in future intervention studies. Clin Psychol Rev. 2018;59: 78–100. pmid:29167029
- 31. Burton NW, Pakenham KI, Brown WJ. Evaluating the effectiveness of psychosocial resilience training for heart health, and the added value of promoting physical activity: a cluster randomized trial of the READY program. BMC Public Health. 2009;9: 427. pmid:19930615
- 32. Burton NW, Pakenham KI, Brown WJ. Feasibility and effectiveness of psychosocial resilience training: a pilot study of the READY program. Psychol Health Med. 2010;15: 266–77. pmid:20480432
- 33. Hawkes AL, Chambers SK, Pakenham KI, Patrao TA, Baade PD, Lynch BM, et al. Effects of a telephone-delivered multiple health behavior change intervention (CanChange) on health and behavioral outcomes in survivors of colorectal cancer: a randomized controlled trial. J Clin Oncol. 2013;31(18): 2313–21. pmid:23690410
- 34. Hawkes AL, Pakenham KI, Chambers SK, Patrao TA, Courneya KS. Effects of a multiple health behavior change intervention for colorectal cancer survivors on psychosocial outcomes and quality of life: a randomized controlled trial. Ann Behav Med. 2014;48(3): 359–70. pmid:24722960
- 35. Ryan A, Pakenham KI, Burton N. A pilot evaluation of a group acceptance and commitment therapy informed resilience training program for people with diabetes. Aust Psychol. 2019;1–12.
- 36. Hayes SC, Luoma JB, Bond FW, Masuda A, Lillis J. Acceptance and commitment therapy: Model, processes and outcomes. Behav Res Ther. 2006;44: 1–25. pmid:16300724
- 37. Kashdan TB, Rottenberg J. Psychological flexibility as a fundamental aspect of health. Clin Psychol Rev. 2010;30(7): 865–878. pmid:21151705
- 38. Marshall EJ, Brockman RN. The Relationships Between Psychological Flexibility, Self-Compassion, and Emotional Well-Being. J Cogn Psychother. 2016;30(1): 60–72. pmid:32755906
- 39. Gloster AT, Meyer AH, Lieb R. Psychological flexibility as a malleable public health target: Evidence from a representative sample. J Contextual Behav Sci. 2017;6(2): 166–171.
- 40. Stabbe OK, Rolffs JL, Rogge RD. (2019). Flexibly and/or inflexibly embracing life: Identifying fundamental approaches to life with latent profile analyses on the dimensions of the Hexaflex model. J Contextual Behav Sci. 2019, 12: 106–118.
- 41. Powers MB, Zum Vorde Sive Vording MB, Emmelkamp PM. Acceptance and commitment therapy: a meta-analytic review. Psychother Psychosom. 2009;78(2): 73–80. pmid:19142046
- 42. Ruiz FJ. A review of Acceptance and Commitment Therapy (ACT) empirical evidence: Correlational, experimental psychopathology, component and outcome studies. Int Psicol Ter Psicol. 2010;10(1): 125–162.
- 43. Swain J, Hancock K, Hainsworth C, Bowman J. Acceptance and commitment therapy in the treatment of anxiety: a systematic review. Clin Psychol Rev. 2013;33(8): 965–978. pmid:23999201
- 44. A-Tjak JG, Davis ML, Morina N, Powers MB, Smits JA, Emmelkamp PM. A meta-analysis of the efficacy of acceptance and commitment therapy for clinically relevant mental and physical health problems. Psychother Psychosom. 2015;84(1): 30–36. pmid:25547522
- 45. Spijkerman MP, Pots WT, Bohlmeijer ET. Effectiveness of online mindfulness-based interventions in improving mental health: A review and meta-analysis of randomised controlled trials. Clin Psychol Rev. 2016;45: 102–114. pmid:27111302
- 46. Dindo L, Van Liew JR, Arch JJ. Acceptance and Commitment Therapy: A Transdiagnostic Behavioral Intervention for Mental Health and Medical Conditions. Neurotherapeutics. 2017;14(3): 546–553. pmid:28271287
- 47. Graham CD, Gouick J, Krahé C, Gillanders D. A systematic review of the use of Acceptance and Commitment Therapy (ACT) in chronic disease and long-term conditions. Clin Psychol Rev. 2016;46: 46–58. pmid:27176925
- 48. Gloster AT, Walder N, Levin ME, Twohig MP, Karekla M. The empirical status of acceptance and commitment therapy: A review of meta-analyses. J Contextual Behav Sci. 2020;18: 181–192.
- 49. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013;50(5): 587–92. pmid:23159157
- 50. Giovannetti AM, Quintas R, Tramacere I, Giordano A, Confalonieri P, Messmer Uccelli M, et al. A resilience group training program for people with multiple sclerosis: Results of a pilot single-blind randomized controlled trial and nested qualitative study. PLoS One. 2020;15(4):e0231380. pmid:32271833
- 51. Giovannetti AM, Messmer Uccelli M, Solari A, Pakenham KI. Evaluation of a program for training psychologists in an acceptance and commitment therapy resilience intervention for people with multiple sclerosis: a single-arm longitudinal design with a nested qualitative study. Disabil Rehabil. 2022 Feb 1:1–13. pmid:35100924
- 52. Giovannetti AM, Solari A, Pakenham KI. Effectiveness of a group resilience intervention for people with multiple sclerosis delivered via frontline services. Disabil Rehabil. 2021;18:1–11. pmid:34406895
- 53. Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007;7: 16. pmid:17573961
- 54. Grant S, Mayo-Wilson E, Montgomery P. CONSORT-SPI 2018 Explanation and Elaboration: guidance for reporting social and psychological intervention trials. Trials. 2018;19: 406. pmid:30060763
- 55. Chan AW, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 Explanation and Elaboration: Guidance for protocols of clinical trials. BMJ. 2013;346: e7586. pmid:23303884
- 56. Calvert M, Kyte D, Mercieca-Bebber R, Slade A, Chan AW, King MT, et al. Guidelines for Inclusion of Patient-Reported Outcomes in Clinical Trial Protocols: The SPIRIT-PRO Extension. Jama. 2018;319(5): 483–494. pmid:29411037
- 57. Polman CH, Reingold SC, Banwell B, Clanet M, Cohen JA, Filippi M, et al. Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann Neurol. 2011;69: 292–302. pmid:21387374
- 58. Ivers NM, Halperin IJ, Barnsley J, Grimshaw JM, Shah BR, Tu K, et al. Allocation techniques for balance at baseline in cluster randomized trials: a methodological review. Trials. https://www.ncbi.nlm.nih.gov/pubmed/228538202012;13: 120. pmid:22853820
- 59.
Evans S, Royston P, Day S. Minim: Allocation by minimization in clinical trials. 2017 May 4 [cited 01 December 2021] In: John Martin Bland page [Internet] University of York. Available from: https://www-users.york.ac.uk/~mb55/guide/minim.htm.
- 60. Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology. 1983;33: 1444–52. pmid:6685237
- 61.
Hayes SC, Strosahl KD, Wilson KG. Acceptance and Commitment Therapy, The Process and Practice of mindful change. (2nd ed.). The Guildford Press. New York/London 2012.
- 62.
De Chirico G. Corso di training autogeno. Per superare ansie, paure, disturbi psicosomatici e per smettere di fumare. Red Edizioni 2010.
- 63. O’Neill L, Latchforda G, McCrackenc LM, Graham CD. The development of the Acceptance and Commitment Therapy Fidelity Measure (ACT-FM): A delphi study and field test. J Contextual Behav Sci. 2019;14: 111–118.
- 64. Connor KM, Davidson JRT. Development of a new resilience scale: The Connor-Davidson resilience scale (CD-RISC). Depress Anxiety. 2003;18(2): 76–82. pmid:12964174
- 65. Davidson JR, Payne VM, Connor KM, Foa EB, Rothbaum BO, Hertzberg MA, et al. Trauma, resilience and saliostasis: effects of treatment in post-traumatic stress disorder. Int Clin Psychopharmacol. 2005;20(1): 43–48. pmid:15602116
- 66. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67: 361–370. pmid:6880820
- 67. Crawford JR, Henry JD. The positive and negative affect schedule (PANAS): construct validity, measurement properties and normative data in a large non-clinical sample. Br J Clin Psychol. 2004;43(Pt 3): 245–265. pmid:15333231
- 68. Vickrey BG, Hays RD, Harooni R, Myers LW, Ellison GW. A health-related quality of life measure for multiple sclerosis. Qual Life Res. 1995;4: 187–206. pmid:7613530
- 69. Giordano A, Pucci E, Naldi P, Mendozzi L, Milanese C, Tronci F, et al. Responsiveness of patient reported outcome measures in multiple sclerosis relapses: the REMS study. J Neurol Neurosurg Psychiatry. 2009;80(9): 1023–8. pmid:19443471
- 70. Johnson JA, Coons SJ, Ergo A, Szava-Kovats G. Pharmacoeconomics. 1998;13: 421–433. pmid:10178666
- 71. Lamers SM, Westerhof GJ, Bohlmeijer ET, ten Klooster PM, Keyes CL. Evaluating the psychometric properties of the mental health continuum-short form (MHC-SF). J Clin Psych. 2011;67: 99–110.
- 72. Rolffs JL, Rogge RD, Wilson KG. Disentangling Components of Flexibility via the Hexaflex Model: Development and Validation of the Multidimensional Psychological Flexibility Inventory (MPFI). Assessment. 2018; 25(4): 458–482. pmid:27152011
- 73. Landi G, Pakenham KI, Giovannetti AM, Presti G, Boccolini G, Cola A, et al. Italian Validation of the Italian Multidimensional Psychological Flexibility Inventory (MPFI). J Contextual Behav Sci. 2021;21: 57–65.
- 74. Lin YY, Rogge RD, Swanson DP. (2020). Cross-cultural flexibility: Validation of the traditional Mandarin, simplified Mandarin, and Japanese translations of the Multidimensional Psychological Flexibility Inventory. J Contextual Behav Sci, 15: 73–84.
- 75. Rogge RD, Daks JS, Dubler BA, Saint KJ. (2019). It’s all about the process: Examining the convergent validity, conceptual coverage, unique predictive validity, and clinical utility of ACT process measures. J Contextual Behav Sci, 14: 90–102.
- 76. Seidler D, Stone B, Clark BE, Koran J, Drake CE. Evaluating the factor structure of the Multidimensional Psychological Flexibility Inventory: An independent replication and extension. J Contextual Behav Sci. 2021;17: 23–31.
- 77. Costantini M, Musso M, Viterbori P, Bonci F, Del Mastro L, Garrone O, et al. Detecting psychological distress in cancer patients: validity of the Italian version of the Hospital Anxiety and Depression Scale. Support Care Cancer. 1999;7: 121–7. pmid:10335929
- 78. Terracciano A, McCrae RR, Costa PT Jr. Factorial and construct validity of the Italian Positive and Negative Affect Schedule (PANAS). Eur J Psychol Assess. 2003;19(2): 131–141. pmid:20467578
- 79. Solari A, Filippini G, Mendozzi L, Ghezzi A, Cifani S, Barbieri E, et al. Validation of Italian multiple sclerosis quality of life 54 questionnaire. J Neurol Neurosurg Psychiatry. 1999;67: 158–62. pmid:10406981
- 80. Balestroni G, Bertolotti G. EuroQol-5D (EQ-5D): an instrument for measuring quality of life. Monaldi Arch Chest Dis. 2012;78: 155–159. pmid:23614330
- 81. Petrillo G, Capone V, Caso D, Keyes CLM. The Mental Health Continuum–Short Form (MHC–SF) as a Measure of Well-Being in the Italian Context. Soc Indic Res. 2015;121: 291–312.
- 82. Donner A, Klar N. Statistical considerations in the design and analysis of community intervention trials. J Clin Epidemiol. 1996;49(4): 435–439. pmid:8621994
- 83. Leyrat C, Morgan KE, Leurent B, Kahan BC. Cluster randomized trials with a small number of clusters: which analyses should be used? Int J Epidemiol. 2018;47(1): 321–331. pmid:29025158
- 84. Huque MH, Moreno-Betancur M, Quartagno M, Simpson JA, Carlin JB, Lee KJ. Multiple imputation methods for handling incomplete longitudinal and clustered data where the target analysis is a linear mixed effects model. Biom J. 2020;62(2): 444–466. pmid:31919921
- 85. Öst LG. The efficacy of Acceptance and Commitment Therapy: an updated systematic review and meta-analysis. Behav Res Ther. 2014;61: 105–21. pmid:25193001
- 86. Pakenham KI, Landi G, Boccolini G, Furlani A, Grandi S, Tossani E. The moderating roles of psychological flexibility and inflexibility on the mental health impacts of COVID-19 pandemic and lockdown in Italy. J Contextual Behav Sci. 2020;17: 109–118. pmid:32834969
- 87. Landi G, Pakenham KI, Boccolini G, Grandi S, Tossani E. Health Anxiety and Mental Health Outcome During COVID-19 Lockdown in Italy: The Mediating and Moderating Roles of Psychological Flexibility. Front Psychol. 2020; pmid:32982888
- 88. Qiu J, Shen B, Zhao M, Wang Z, Xie B, Xu Y. A nationwide survey of psychological distress among Chinese people in the COVID-19 epidemic: implications and policy recommendations. Gen Psychiatr. 2020;33(2): e100213. pmid:32215365