Aim 2 –process evaluation–patient satisfaction interviews and fidelity monitoring.

Patient satisfaction interview recruitment. Patients will be recruited to complete a brief post-intervention satisfaction interview via IRB-approved flyers distributed by the MROT team members, containing a phone number to call if participants are interested in participating in the interview. MROT staff will also ask patients if they would like to sign a release of information to allow the MROT team member to give patient contact information directly to the research team. Eligibility will be assessed using a brief screening questionnaire conducted over the phone. Eligibility criteria include: being over 18 years of age, having been seen for an opioid-related visit (overdose or primary/secondary diagnosis of opioid use disorder) in an intervention hospital in the last 30 days, and having met with an MROT staff member during that visit.

Study staff are located in northern and southern Nevada. Eligibility screening and interview scheduling will occur in the southern Nevada office, although interviewers are available in both locations to conduct in-person interviews at a location of the patient’s choosing (i.e., a study field office, hospital conference room, or semi-private location agreed to by the patient). After determining eligibility, an interview time and location will be established in a central paper appointment scheduler that will be maintained in the southern Nevada study office to avoid breaches of confidentiality and/or confusion in scheduling. If the patient lives in northern Nevada, the local staff member will be notified of the appointment date and time via phone. One day prior to, or the day of, the appointment a call will be made to the participant to confirm the appointment.

Patient satisfaction interview procedures. All participants will provide written informed consent according to the IRB-approved protocol. After the informed consent procedure, data will be collected via in-person interviews utilizing Qualtrics® software on a laptop computer or tablet that has been certified by the Federal Information Processing Standard (FIPS) Publication 140–2 security standard. Because of the mixed methods nature of the interview, in which we collect both quantitative survey data and qualitative narrative, the interviews will be recorded in their entirety using the device’s internal microphone (with permission from the participant). Recorded interviews will be transcribed verbatim by study staff and subjected to Quality Assurance procedures to ensure accuracy. Participants will be compensated $50.00 cash for their time and effort.

Patient satisfaction interview measures. The collection and analysis of quantitative (QUAN) and qualitative (QUAL) data will occur simultaneously in the same interview instrument. The surveys will begin with QUAL items designed to elicit narrative descriptions of the event of interest (i.e., meeting a mobile team in the hospital and all subsequent interactions following that event), followed with more specific QUAN questions including enrollment in social services, medication assisted treatment choices, subsequent overdoses and/or witnessing overdoses, and demographic information. In addition, QUAN scales such as MAT stigma, satisfaction with the encounter, and social support are included. We order the surveys in this way (QUAL then QUAN) because in other research we have identified a tendency for participants to provide shorter and less illustrative responses to QUAL questions if they have been asked to answer closed-ended QUAN questions at the beginning of the interview. We have also observed that the QUAL questions facilitate rapport-building between interviewer and respondent, which may increase the validity of responses to the QUAN questions.

Patient satisfaction interviews analysis. QUAL and QUAN data from the patient satisfaction interviews will be analyzed simultaneously, compared or related to each other, and interpreted together. First, we will describe the distribution of the QUAN variables using measures of frequency, central tendency, and dispersion. We will examine the psychometric properties of the multi-item MAT stigma construct by calculating Cronbach’s alpha to assess its reliability. Second, transcripts from the patient satisfaction interviews will be analyzed using an iterative thematic approach. We will review individual transcripts and interviewer field notes, making memos that document initial impressions and begin to synthesize observations. Based on this initial read, a set of initial “open codes” will be developed to identify a priori any emergent themes and will be applied to the entire set of notes and transcripts using Atlas.ti software for organization of thematic content. The codes will be reviewed and condensed into a set of hierarchically-arranged topics, and data will be extracted from Atlas.ti according to those codes. A second review of the coded data will identify any additional sub-themes or codes that should be added. Throughout the process, memos will be updated to document emergent understandings of the connections between themes.

Aim 2 –process evaluation–process data & fidelity monitoring.

Process evaluation and fidelity monitoring data collection and measures. Intervention process data will be acquired from OPHIE as part of our data sharing agreement and directly from the MROTs. All data will be de-identified and tabulated. Fidelity monitoring data will be collected for descriptive purpose using two methods: MROT members will fill out a self-report fidelity assessment in Qualtrics®, and a trained research assistant will conduct fidelity observations of 10% of the intervention sessions using a structured observation checklist. The self-report and fidelity monitoring checklist includes an itemized list of intervention components, assessment of quality and proportion delivered of the brief negotiated interview, ability to provide patient with requested resources (including take home naloxone and ability to take next steps for treatment), any plans for future communication with the peer, and any additional notes.

Process evaluation and fidelity monitoring data analysis. Process data will be tabulated over the 9-month intervention period, including: the number of opioid overdose patients presenting in participating EDs, the number of patients who have made contact with the MROT staff, the number of patients receiving overdose education and naloxone from MROT staff, the number of referrals made to treatment services and/or MAT, the duration of MAT participation if enrolled, and the number of subsequent fatal and nonfatal overdoses among MROT patients. Fidelity monitoring data will be described using frequencies and measures of central tendency and dispersion.

Supplemental study activities.

In addition to the primary outcomes and process evaluation described above, the MERIT study will: 1) evaluate the feasibility and acceptability of an alternative MROT model for rural/frontier EDs that do not have the same access to resources (including peer recovery support specialists, methadone clinics, and buprenorphine prescribers) as their urban counterparts; and 2) use an Implementation Science approach to examine the barriers and facilitators of successful program implementation in the six intervention hospitals participating in the outcomes evaluation trial.

Theoretical approach for supplemental activities.

We will use Diffusion of Innovation (DOI) theory to inform the feasibility and acceptability research among the rural/frontier hospitals. This theory was developed based on empirical work demonstrating that there is a consistent pattern of adoption of new ideas in a system or organization [5]. DOI describes five characteristics of an innovation that characterize its adoptability: relative advantage, complexity, compatibility, trialability, and observability [6]. Using this framework, we will examine how the MROT intervention is described and received by ED staff, MROT team staff, key stakeholders in the hospitals and state agencies, and patients, with a focus on perceived usefulness (relative advantage) and ease of use (complexity). These two characteristics are especially important as they will provide the context needed to refine the intervention for future dissemination. Finally, potential barriers to adoption of the intervention will be explored as perceived by those inside the hospital organizations, people who use opioids, and MROT staff.

To inform the Implementation Science aim, we will use the Consolidated Framework for Implementation Research (CFIR). CFIR includes five domains and 39 constructs that are salient for implementing a successful innovation [7]. We will focus on those domains and constructs that were identified as critical to adoption in the formative phase of this research, including intervention characteristics (relative advantage, adaptability, complexity), inner setting (culture, relative priority, readiness for implementation, and leadership engagement), and process (planning, engaging, champions, and execution) [4].

Data collection for supplemental activities.

Feasibility and acceptability interviews will be conducted with ED providers and MROT staff. ED providers will be recruited through IRB-approved flyers that will be disseminated through employee newsletters, professional organizations, one-on-one contact, and postings in the hospital’s employee break rooms. We will attempt to diversify the sample in terms of sex/gender and professional roles. MROT staff will be recruited through one-on-one contact. Implementation science interviews will be conducted with stakeholders who have played a prominent role in establishing MROT teams. Stakeholders will be recruited through a non-descript email. For all supplemental activity interviews, the IRB has approved a waiver of documentation of consent because the only piece of identifying information linking a participant to the study would be a signed consent form. Study staff will review an IRB-approved Information Sheet with participants and ask them to explicitly provide their verbal consent before proceeding with study activities.

ED and MROT surveys will include QUAL questions that will ask the respondents to describe how the MROT integrates into existing ED operations and QUAN questions that assess relative advantage, complexity, and compatibility of the MROTs in their current organizational structure. We will also ask them to rate their level of awareness, enthusiasm, and support for the MROT program. Stakeholder interviews will include QUAL questions that help frame the broader community context that existed before and during the MROT team rollout and the role individual stakeholders played in influencing the rollout. In addition, QUAN questions will assess whether MROTs are perceived as an effective strategy for the state response to the opioid overdose crisis.

To further examine the structural, cultural, and organizational factors that may affect MROT implementation, we will conduct participant observation in hospital EDs. Hospitals may refuse to participate in the participant observation; refusal will not affect provision of mobile team services. Participant observation will identify structural and cultural factors that influence the flow of information within hospital EDs. Ethnographic annotations will be derived using undisguised participant observations that include unstructured interviews, notes based on observations and interactions, and documentation of observed surroundings such as sketches of the environment itself by research personnel and/or MROT staff. Subjects to be observed include visual communication within workspaces (for example notices, signage, data display screens, noticeboards, and white boards), one-on-one interpersonal verbal communication amongst staff and between staff and patients (for example speech, gesture, affect, and tone), and stimuli in workspaces that have the potential to affect reception of information (for example sound, foot traffic, brightness, and spatial arrangement).

These observations and visual communication artifacts will be documented in field notes that will be coded and analyzed using visual data methodologies. Specifically, discursive visual analysis methods will be used to examine the production, use, and rhetorical composition of observed spaces by observed subjects [8]. Analyses will focus on identifying a) how information is disseminated broadly by the ED staff regarding the mobile teams in both physical locations in the hospital and through person-to-person communication efforts; b) barriers to receiving information about mobile teams in EDs; and c) opportunities for improved communication of information about mobile teams.