Process evaluation of an academic-community-government partnership to reduce liver diseases attributable to hepatitis B virus

Abstract

Background

Racial/ethnic minorities have higher incidence and mortality rates of liver cancer, or hepatocellular carcinoma, than non-Hispanic Whites. As such, the Washington-Baltimore Metropolitan Area Hepatitis B Virus (WB-HBV) Demonstration Project, a community-based participatory research (CBPR)-driven academic-community-government (ACG) partnership, was established in 2019 to address disparities and implement strategies to improve the HBV screening and vaccination infrastructure for at-risk communities. CBPR is a partnership of community members, organizational leaders, and academic researchers with a common aim to collectively share and contribute their input at every phase of the project.

Herein, we describe the process evaluation of the WB-HBV Project and extract themes and insights to benefit future ACG partnerships and community-engaged research. The process evaluation has been conducted to determine whether CBPR-driven partnership and programmatic activities have been implemented as intended and have resulted in building expanded research capacity for future ACG partnership HBV community-level initiatives.

Methods

A WB-HBV Project Task Force was convened and comprised of eight organizations: four community organizations, three government organizations, and one academic institution. Through a mixed-methods process evaluation, an online survey and key informant interviews were conducted to provide context for program implementation barriers and facilitators. Descriptive statistics were conducted, and interviews were recorded, transcribed, and thematically coded.

Results

The survey was completed by 14 of 20 partnership members (70.0%): two academic, eight community, and four government members. Partnership members showed general agreement across 14 domains: organization and structure of meetings; trust; decisions; impact; general satisfaction; strategic planning; ACG policy impact; community-based participatory research and government; participation in meetings; assessment of participation; partnership operations and capacity; communication; challenges/limitations associated with ACG involvement; and benefits compared to challenges associated with ACG involvement. Qualitative interviews were conducted with 15 of the 20 members (75.0%): two academic, nine community, and four government members. Four themes emerged: partnership involvement, project goals and accomplishments, project challenges and barriers, and partnership involvement in government or policy.

Conclusions

The process evaluation presents insights into developing strategies to enhance partnership functioning and increase the ability of present and future ACG partnerships to improve community health outcomes.

Background

Community-based participatory research (CBPR) is a partnership of community members, organizational leaders, and academic researchers with a common aim to collectively share and contribute their knowledge and expertise to improve the health of community members [1]. Community members’ active and continued engagement in CBPR is crucial, especially because of challenges conducting research in underserved communities [2]. CBPR fosters equitable participation of each stakeholder from the academic, community, and government settings to enable collaborative changes that will improve the health of community members [3]; CBPR has been widely used in research and has been shown as an effective method to reduce cancer disparities [4]. However, challenges in building, engaging, and sustaining CBPR partnerships often exist [4]. Overcoming these challenges requires assessing program implementation barriers and facilitators among the collaborative partnership members working to improve community health.

Liver cancer, or hepatocellular carcinoma (HCC), is a leading cause of death worldwide, and approximately 24,000 men and 10,000 women are diagnosed with HCC each year in the US [5,6,7]. According to the Centers for Disease and Prevention (CDC), approximately 18,600 men and 9000 women die from HCC each year [7]. The risk for HCC increases with chronic hepatitis B virus (HBV) or hepatitis C virus (HCV) infection. Despite vaccination and treatment options, respectively, for HBV and HCV infections, racial/ethnic minorities have higher incidence and mortality rates than non-Hispanic Whites [5]. In the Washington-Baltimore Metropolitan Area (WBMA) from 2009 to 2015, the prevalence rates of HBV and HCV, respectively, were 6.1 and 3.8% among Asian-born immigrants and 3.7 and 2.8% for African-born immigrants [6]. Since most infected individuals show little to no symptoms until their liver disease is well advanced, they are often diagnosed with late-stage cancer that results in low survival rates and high mortality rates [5,6,7]. Due to the asymptomatic nature of HCC, it is crucial to screen and prevent complications from HCC among foreign-born individuals who have migrated from countries where hepatitis viruses are endemic.

The Washington-Baltimore Metropolitan Area Hepatitis B Virus (WB-HBV) Demonstration Project is an integrated, multijurisdictional community coalition that aims to deliver prompt, responsive, and efficient care for immigrants from countries where HBV infection is endemic (e.g., Asia and Africa). The overall objective of this collaborative academic-community-government (ACG) partnership is to implement strategies and interventions to improve HBV screening and vaccination infrastructure for at-risk communities in Washington, District of Columbia (DC); Maryland; and Virginia. It covers a cross-state area populated by residents identified by the CDC as being most at-risk for and affected by this disease. The WB-HBV Project Task Force comprises the National Institute of Diabetes and Digestive and Kidney Diseases within the National Institutes of Health, four community organizations (inclusive of primary care physicians, a virologist, and an infectious disease specialist), three local health departments, and the George Washington University research team. To assess program implementation barriers and facilitators among the collaborative partnership members, an external evaluation team was convened to develop and execute a process evaluation plan. The purpose of this article is to (a) describe the process evaluation of an ACG partnership created to address HBV health disparities in the WBMA, and (b) extract themes and insights to benefit future ACG partnerships and the field of CBPR.

Methods

A WB-HBV Project Task Force was convened and comprised of eight local organizations: four community organizations, three government organizations, and one academic institution. Through purposeful sampling, these organizations were selected based on their historical and current efforts in providing HBV outreach, education, screening or testing, and linkage to care and treatment services to at-risk communities, particularly among the foreign-born population, in the WBMA. This cross-jurisdictional collaborative partnership was intentionally organized to develop a sustainable model to allow for HBV health information exchange between providers in the WBMA.

To conduct process evaluation, an external evaluation team approached task force members using an online survey and key informant interviews. This mixed-methods approach using closed and open-ended questions was implemented to quantitatively and qualitatively provide context for program implementation barriers and facilitators among members of the WB-HBV Project Task Force. The external evaluation team consisted of 3 individuals who were not directly involved in the implementation of the WB-HBV Project.

Data collection and measures

Brief partnership interview survey

The quantitative evaluation survey was adapted from Israel and colleagues [8]. The original survey consisted of 110 items that assessed 15 domains. Items assessed included general satisfaction, trust, operations and capacity, and organization and structure of meetings [8]. To reduce participant burden, the external evaluation team identified and adopted 69 items from the original survey. A confidential online instrument assessed the organization and structure of meetings, trust, decisions, impact, general satisfaction, strategic planning, ACG policy impact, CBPR and government, participation in meetings, assessment of participation, partnership operations and capacity, communication, benefits of ACG involvement, challenges/limitations associated with ACG involvement, and benefits compared to challenges associated with ACG involvement. The majority of the responses were on a 5-point Likert scale: (1) strongly agree, (2) agree, (3) neutral, (4) disagree, and (5) strongly disagree. Some responses were on a 4-point Likert scale: (1) increased, (2) stayed same, (3) decreased, and (4) don’t know or (1) a lot, (2) moderate amount, (3) not much, and (4) don’t know.

Key informant interviews

Upon survey completion, participating members of the WB-HBV Project Task Force were contacted by a graduate research assistant to schedule their subsequent follow-up key informant interview. The interview guide consisted of open-ended questions, also derived from Israel and colleagues [9], on capacity building of task force members to participate in CBPR partnerships (Fig. 1) [1, 3].

Fig. 1

Interview guide

Interview questions focused primarily on core partnership content areas, which included membership, goals/accomplishments for the first year, major barriers/challenges for the first year, goals for Project Years 2 and 3, barriers/challenges for Project Years 2 and 3, organizational impact, government/policy, and partnership. Subtopics of further interest to the evaluation subcommittee included partnership history (e.g., role in partnership and governance; role of community members in the research process), co-learning experience (e.g., goals and barriers/limitations for Project Years 1, 2, and 3, benefits and challenges of working in the partnership), and perceptions of the ACG partnership’s effectiveness to address community needs. All interviews were conducted either via phone or WebEx by a trained research assistant and lasted no longer than 1 h.

Data analyses

Survey interview questions were administered online via Research Electronic Data Capture (REDCap), a secure web-based application designed to support data capture for research studies and hosted by the Clinical and Translational Science Institute at Children’s National [10]. The online survey data were downloaded into Statistical Package for the Social Sciences (SPSS) v.27.0, from which descriptive statistics (e.g., frequency distribution and comparison of means) were generated [11].

Audio-recorded interviews were transcribed verbatim (with members’ permission) for further review and analysis. Each transcript was thematically coded using QSR International’s NVivo software (version 12 Plus) [12] by at least two researchers and cross-checked for agreement. A taxonomy of emergent themes was developed and shared between all reviewers (DL, AC, MJJ, RH, AS) as an iterative and collaborative process. Content analysis was used to examine variations in the data to further refine the coding scheme [13, 14]. Themes were organized into overarching domains and compiled with representative quotations. Discrepancies were resolved through a process of constant comparison until inter-rater reliability was reached (Kappa coefficient > .80).

Results

16 of 20 partnership members participated in the current study, completing either both the survey and the interview or at least one of these two components. There was one participant who completed the survey but not the interview, and there were two participants who completed the interview but not the survey.

Results from the survey data

The survey was completed by 14 of 20 partnership members (70.0%): two academic, eight community, and four government members. Collaborative partnership members’ perspectives on partnership-building and research across the following 14 domains are presented in Table 1: organization and structure of meetings; trust; decisions; impact; general satisfaction; strategic planning; ACG policy impact; CBPR and government; participation in meetings; assessment of participation; partnership operations and capacity; communication; challenges/limitations associated with ACG involvement; and benefits compared to challenges associated with ACG involvement.

Table 1 Collaborative partnership members’ perspectives on partnership-building and research (N = 14)

Organization and structure of meetings

Members agreed that ACG partnership meetings were generally useful, well-prepared, and organized; many felt comfortable voicing their opinions at the meetings. Although individuals from the academic and government organizations felt that much was accomplished at these meetings and expressed neutrality regarding the statement, “I wish we spent more time at partnership meetings hearing about and discussing ACG projects,” community members expressed the need for increased dialogue but were neutral as to whether much was completed during the sessions.

Trust

Participants agreed that the relationships among ACG members extended to include member organizations beyond the individuals at the table. They expressed comfort in requesting assistance, introducing new ideas, speaking frankly, respecting each other’s viewpoints, and being heard. Academic members reflected a slight increase in comfort expressing opinions at meetings and confidence in ACG members collaborating; governmental members experienced increased trust between members over the year prior to the survey.

Decisions

While members expressed satisfaction with ACG decision-making and felt everyone contributed, they were neutral about the time the partnership took to reach decisions.

Impact

Participants felt the partnership was effective regarding its goals, benefits to the community, and improvements in health outcomes in the WBMA. They believed that participation increased understanding of the other organizations, knowledge on health disparities and the social determinants, and capacity for CBPR.

General satisfaction

Members were satisfied with the task force’s activities and progress, the types of projects implemented, attention to the sustainability of the partnership (as measured across the following three dimensions essential to sustainability: (1) relationships and commitments from partnership members, (2) knowledge, capacity, and values of the partnership, and (3) funding, staff, programs, policy changes, and partnership itself [15]), and capacity building of partners in the work. Community partners felt they had somewhat adequate knowledge of the ACG’s logistics and wanted to provide more input regarding the allocation of resources, while government partners were neutral. Community partners were less satisfied with the ACG’s translation of research and evaluation into information and programs to improve health in the WBMA.

Strategic planning

The sustainability of the ACG partnership depended on the task force’s strategic planning processes. Participants agreed that the task force’s strategic planning resulted in developing (1) concrete goals and objectives, and (2) appropriate strategies to accomplish them.

ACG policy impact

Participants felt the ACG was effective in informing policymakers about the ACG and its initiatives and in providing support for relevant policy issues. Community and government members felt that the ACG was less effective at translating findings into policy-relevant materials.

CBPR and government

Members strongly agreed on the importance of informing key government officials about the partnership and its initiatives. Members reported that community interests were well represented in ACG activities; however, opportunities to represent the ACG were somewhat limited for community members.

Participation in meetings

When asked how often participants attended ACG meetings over the last year, partnership members from the academic setting reported attending approximately seven to nine meetings; members from government and community reported attending only one to three meetings.

Assessment of participation

Most participants, especially academic members, indicated satisfaction with their participation level in the ACG partnership. Community members were more likely than their academic and government counterparts to believe that they influenced the work of the ACG partnership.

Partnership operations and capacity

All members agreed that the ACG partnership thought strategically, was well-managed, and had a clear vision of what it aspired to achieve, and that the vision had concrete, measurable goals that members aimed to achieve. Members also felt that the ACG follows its own CBPR principles, effectively represents the diversity of the communities, and represents community interests in ACG activities. However, academic partners, compared to other members, differed on whether the ACG partnership has a clear vision of what it aspires to achieve and follows its own CBPR principles. Government members were more likely to respond favorably to the majority of the partnership operations and capacity measures. Across the nine measures within this domain, participants most enthusiastically agreed that ACG partnership members took responsibility for getting the work done, and their capacity to work well together had increased in the past year.

Communication

Participants agreed that ACG members effectively communicated with one another during and outside of partnership meetings; members from the academic and government settings, however, expressed slightly stronger agreement than their counterparts from the community.

Benefits of ACG involvement

Participants unanimously agreed that various benefits were gained from the ACG partnership. Academic members were more likely than their partnership colleagues to agree that “an increase in recognition and respect for their institution in the WBMA” occurred, while community and government members were more likely to report “opportunities to work with communities with whom the partnering organizations have previously had minimal contact with” and “development of new collaborative relationships across the partnering organizations” as benefits of their ACG involvement, respectively.

Challenges/limitations (associated with ACG involvement)

Academic members agreed that the ACG partnership activities addressed their overall goals and interests, while community and government members’ feelings on this topic remained more neutral. Although academic and government members were more likely to agree that their opinions (and their organizations’ opinions) were valued, and substantial funding existed for their respective organizations to participate in the partnership, community partners continued to express neutrality in their responses on these measures.

Benefits compared to challenges (associated with ACG involvement)

From organizational and personal perspectives, academic and community members were more likely than their government counterparts to agree that the benefits of participating in the ACG partnership outweighed any challenges encountered to date.

Results from the in-depth interviews

Qualitative interviews were conducted with 15 of the 20 partnership members (75.0%): two academic, nine community, and four government members. Interviews lasted from 30 to 45 minutes. As presented in Table 2, four main themes were identified relating to the ACG partnership: (1) partnership involvement, (2) project goals and accomplishments, (3) project challenges and barriers, and (4) partnership’s involvement in government or policy.

Table 2 Themes and codes

Partnership involvement

Role and involvement

Community partners were tasked to screen individuals for HBV surface antibodies, HBV surface antigens, and HBV core antibodies. Among vaccinated individuals, community partners were also tasked to confirm which HBV vaccine was administered - either the 2-dose Heplisev, the 3-dose Engerix B, or the Recombivax HB. Health department partners were expected to help recruit participants, interpret data collected, and support the partnership activities. Academic partners were tasked to complete IRB application and protocol development as well as manage all community partner data collection, aggregation, and reporting to the federal funder. In addition, evaluation teams were expected to evaluate process and outcome measures.

Benefits of the ACG Partnership

Participants described multiple benefits of the ACG partnership. The primary benefits described included the synergistic relationship of the organizations to better address HBV disparities in the WBMA community and the increased communication with other organizations due to partnership. A community organization member said:

So this partnership is so crucial, you know, it needs, like this table needs four legs, and it needs this four legs in order for it to work... and so being in this project, it has actually created this big impact. Because, you know, now we can communicate with the Department of Health, we can communicate with the academics, you know, to do publication, so more people are gonna be aware. And so, so the impact is big.

Another policy benefit mentioned was the partnership’s ability to inform policy through information provided to the academic and government organizations. A community member stated, “It’s important to be able to report that information and share it in the academic world. And that those reports and research findings are also shared with the government so they can make better policy decisions.” Moreover, through the partnership, community organizations can work with the academic and government organizations to more efficiently and effectively reach individuals who need help, execute project plans, and disseminate research findings beyond the academic and government settings and into the communities. As two community members pointed out:

Partnerships are to strengthen community relationships and your ability to outreach to the target population, so that actually gave us a better understanding of actually, what ethnicity needs more help. Back then we only mainly focused on the Asian, and African; now we go into the Black community, the LGBT group, and people [who use] drugs.

It’s benefiting them in the sense that they have somebody and people that are well, knowledgeable about things about issues that affect the community, they are benefiting about the fact that we bring to the community and close to them, you know, services and resources that they can use in order to continue to get in control of their own health.

In addition, partnership members affirmed that funding has helped with resources, especially at the start of the Coronavirus 2019 (COVID-19) pandemic, when people were losing jobs and a lot of attention was paid to infectious disease screening and vaccinations. An academic member said, “Several people got funded and other groups as well through this grant during the pandemic. I think it’s really helpful for them. And also, through this grant, more people are getting screened and vaccinated, and linked to care. So that’s, I think it’s substantial health benefits as well.”

Project goals and accomplishments

Year 1 goals

Several goals were mentioned by partnership members, including goals within the ACG partnership and HBV-specific goals. Within the partnership, members wanted to “have more communications,” build capacity, and complete all contractual deliverables. In terms of building capacity, a community member highlighted the importance of having multiple perspectives in the planning and delivery of services:

Yeah, so my understanding when the partnership meetings were set up, it was to engage different stakeholders in the decision-making of being able to outreach, educate, screen link patients to care services relating to hepatitis B, and possibly C, depending on the stakeholders and, you know, where their focus and interest is. So, the stakeholders should have included community-based organizations, the academic institutes, and the government agency, so that there are multifaceted perspectives in the decision-making of that cascade of care.

Regarding goals related to HBV, participants indicated increasing education, screening, vaccination, outreach, resources, and linkage to care. Community members conveyed that providing education and resources and linking patients to care will help lower infections and “bring awareness to the disparities and the different communities that have HBV, and ultimately … a little bit more resources to help these patients.”

Although some members knew the screening goals, others were not too certain what the task force’s project goals were. One commented, “So I’m not really sure what the overall goals were and if they met them or not.” Recommendations included further strengthening the communication between core members of the ACG leadership team (specifically on what the future goals for the partnership should be) and increasing overall engagement and attendance at the ACG meetings.

Accomplishments to-date

Members confirmed that the ACG partnership exceeded their goals; some expressed their appreciation for how quickly team protocols were adjusted to keep participants safe at the start of the COVID-19 pandemic. Screening brought opportunities to link individuals who tested positive to care and vaccinate individuals who tested negative. For example, one community member remarked, “Screening more patients, obviously finding those positives and then I guess for the people that tested negative but who are not immune, then they can get a chance of getting vaccinated and protecting themselves. So those are the things that we were able to accomplish.” Especially for minorities without insurance, this project was important in increasing the overall HBV vaccination rate. One member from a government organization stated, “We also help to improve access to hepatitis B vaccination. We can provide hepatitis B vaccination for adults who are uninsured. And this is a critical population, especially among immigrants new to the country, [who] don’t have insurance yet.” One of the community partnership members further elaborated on how hepatitis prevention is not a priority to the immigrant population: “They have a lot of priority to deal with. So prevention care is definitely not in priority.” Thus, the ACG partnership is taking a big step forward in preventing hepatitis in the immigrant population.

In terms of the partnership, members expressed that the organizations have worked well together by having everyone at the table share, communicate strategies, build capacity, and engage the community. One participant from a government organization emphasized that these types of partnerships can help build future partnerships and collaboration among different organizations working toward a common goal. One partnership member mentioned how obtaining funding was particularly successful in the past year. Another government member discussed how sharing project updates and strategies to overcome barriers was especially helpful:

They give progress reports, they disclose challenges to implementation. I remember, the last one, we talked a lot about our COVID as it impacted the community outreach, testing, and ways that they can go around the outcome, we continue to make sure that we’re providing services, even amid COVID intervention, I think this is really very good.

Future goals for screening

Partnership member goals included meeting set numbers for screening, increasing education and vaccination, having sustainability of screening and vaccination, involving “champions in the community,” addressing HBV birth dose, working with community organizations such as churches, educating the community about COVID-19 as partners adjust to best accommodate patients, and raising awareness about the hybrid model so members can register online to get tested. For example, a community member shared, “I mean, just maybe raise more awareness and let people know that [we have] that hybrid screening model so that, you know, if they’re uncomfortable coming up to a group setting, they can always register online or let them let other people know about our online registration to get tested.” Another participant hoped for a health fair to increase the number of people screened and vaccinated for HBV.

Future goals and recommendations for the ACG Partnership

Goals for the ACG partnership include raising awareness about the high prevalence of HBV among Asian and African immigrants and receiving continued funding from government entities. Two members from community and government organizations (respectively) shared:

I think, more importantly, is rais [ing] awareness with the government agency that’s providing the funding to make sure that funding continues because of the important work that’s being done in the unmet needs that’s happening in viral hepatitis world, especially with hepatitis B. So I think, you know, in that coalition meeting, what needs to be addressed is the understanding that this is still an ongoing disease burden that needs attention... and making sure that, you know, there, there is a sustainable plan, in terms of helping patients realize what the disease burden is, and to raise awareness for prevention.

There might be an opportunity [to] enhance support or funding for hepatitis B. That, I think, would be necessary. And it’s only just now happening for hepatitis C. And this is very recent. I think hepatitis B needs to be looked at similarly.

Other objectives include providing continuity of care for individuals who test HBV positive, strengthening reporting systems and health information exchanges using electronic medical records (EMRs), and addressing HBV as standardized screening. One community member shared:

That’s an important goal, having the EMR systems that are able to do that are fundamental to that process, because it helps clinicians, as they log on, to always be reminded that HBV is part of routine patient care. So that in Year 2 and Year 3, I hope that there’s work to strengthen the EMR component, as well as the health information exchange component, so that both the internal work around HBV and the internal to external reporting and management are both strengthened.

Project challenges and barriers

Overall challenges to-date

Members revealed that coordinating project meeting times has been the biggest challenge, especially for those not working full-time. As one community member shared:

I think one of the biggest challenges of having standing meetings is the time commitment, especially if you’re asking individuals who are not doing this as a full-time job to carve out some time during the work week to meet, I think that could be potentially one of the biggest challenges if this is not their full-time job.

Contrary to their colleagues in the partnership, community members heavily emphasized “how [their] membership on the WB-HBV Project Task Force required/s a considerable time commitment.” Attendance at meetings by community members (1–3 meetings) was considerably lower than attendance by academics (7–9 meetings). Some participants cited “busyness”, “being overly committed with other responsibilities”, and “low staff coverage to attend partnership meetings” as possible reasons. As our project standing meetings were originally agreed upon when the partnership initially formed, pre-covid, it is very likely that new/additional responsibilities naturally arose over time due to the competing/immediate priorities caused by the pandemic (especially for our community partners who were active “in the field”).

In terms of communication between partners, members initially were not made aware of the expectations and did not clearly understand project goals and outcomes. However, once information was shared with them, they had a better grasp on their involvement and contribution to the partnership. For example, one community member revealed:

In the beginning, we were having trouble kind of just gathering all our information because everything just started so we didn’t really know what information was expected from our organization, like number-wise and stuff like that. But once we kind of understood what was expected [it] was easier.

For HBV-related activities (screening, linkage to care, and vaccination), some of the overall challenges mentioned included contacting the target population (including local organizations that serve racial/ethnic minority communities), linking them to care, and working with different communities in the WBMA area. A community member described mistrust as a challenge in HBV-related activities because community organizations are viewed differently from hospitals:

When they come, and of course, we have our banner, our website, and what we offer, but to them, it’s still, “You’re not a hospital. So why should we go there and get our blood drawn [ … ] from someone we don’t know?”

Anticipated future challenges and barriers and recommendations to overcome challenges

Several anticipated challenges were presented by partnership members. One of the biggest challenges was associated with COVID-19, which affected in-person interactions, and therefore limited screening events, community health fairs, and patients’ access to care. A community member shared, “So they’re still not comfortable and that is kind of still a challenge because every year we used to have, like, really big health fairs whereby we are present in person, you know. Anything that you do online is very different from what you have in person.” Members also anticipated challenges to reaching other racial/ethnic minorities communities, such as the Latino or Hispanic populations. Although they have identified these as priority populations, the partnership has not yet established relationships with organizations that predominantly serve these populations. One community member said, “So that was the part that’s new to us, the Hispanic community. We’ve never been able to get into that community. [ …] So that’s what we need, to get into that community more. So I think that’s our barrier.”

Since HBV is incurable and requires prevention resources, challenges within the ACG partnership may include a lack of funding for screening and education or overall funding for HBV-related activities. One member from a government organization stated:

For example, there might be an opportunity to enhance support for funding for hepatitis B. That, I think, would be necessary. And it’s only just now happening for hepatitis C. And this is very recent. I think hepatitis B needs to be looked at similarly. It’s always included, but it’s never received the right attention because it’s not curable.

Another community member mentioned that using evaluation to inform programming, sustain HBV-related activities or programs, and build capacity around health information exchanges or EMRs will be important future work of the ACG partnership that will allow for “a stronger linkage between the community, the community partners and the health department around data.” Members also emphasized that important factors for the ACG partnership are clear communication and transparent goals and outcomes, equitable relationships with partners, and commitment of time.

Partnership’s involvement in government or policy

Current involvement in government or policy

Overall, partnership members indicated that more work should be done to increase HBV funding. One community member shared:

I think there should be more work done with state legislators as well as state viral hepatitis coordinators. But it all really comes down to the federal government and whether or not there’s the budget allocated to address hepatitis B in particular, because I know there is already funding for hepatitis C.

Future recommendations for involvement in government or policy

Partnership members felt that future efforts should include increasing education and HBV vaccine availability and addressing vaccine hesitancy. A community member noted that testing should be widely available to minority populations through similar projects:

Yes, my recommendation is to continue to increase the number of testing, having testing available to help the minority population to get to the point of care. So, if they can have more testing and more partnerships in [the] community that will be a good thing. [Especially] if the work that is being done now is [replicated] to have more people and more partners so that while we are targeting one community, other communities [are] being targeted somewhere else.

In addition, several members emphasized the importance of disseminating findings of the project to legislatures interested in immigrant health, which has been successfully done in the past. One member from the academic organization stated, “I think one thing would be having speakers come in, having us do a presentation with the legislator … and I think when we talk about the dissemination of the findings, making sure we disseminate our findings back to legislators, as they are key stakeholders.”

Discussion

This study provides insight into the facilitators and challenges of ACG partnerships and the importance of conducting partnership evaluations to improve partnership function. The process evaluation of the WB-HBV Demonstration Project highlights the strengths of an ACG partnership in addressing HBV disparities among racial/ethnic minorities and at-risk communities [16]. Most notably, during their first year, the partnership was able to reach its goal of screening 2300 persons by screening 2495 persons, and provided 408 vaccine doses, of its projected goal of 400 doses. Overall, ACG members highlighted the partnership’s positive impact at informing HBV-related policy, reaching project goals and objectives, and overcoming barriers with innovative solutions during the COVID-19 pandemic. Partnership members also described challenges and areas to improve including communicating clear goals and outcomes, task force meetings that are attended by everyone, and overcoming COVID-19-related difficulties in reaching the target population.

Different organizations provided a variety of feedback related to participation: community organization members felt more strongly that they had a positive impact on the partnership, expressed a strong desire to contribute more in the allocation of resources, and described the time required to participate as a major challenge; academic members attended far more meetings (seven to nine versus one to three for other members), felt the most satisfied with their participation, but were more varied on the clarity of the partnership’s vision; government members were the most positive about the partnership’s clear vision and following its own principles, and (with communities members) were least positive about the policy-impacting materials created as a result of the partnership. These variations suggest that members of different organizations have different experiences in, and expectations for, the ACG partnership and introduce the question of how best to structure partnerships so they are more equitable for the members.

The existing literature from recent process evaluations of CBPR partnerships validates (a) the value of the CBPR process for improved outcomes; (b) the importance of community members being involved at every stage of the process; (c) the importance of relationship- and trust-building with community organizations; and (d) the importance of the community to take ownership of the project [17,18,19,20,21,22,23]. The literature also highlights the challenges of communication, inclusiveness, and community involvement to successful CBPR [17,18,19,20,21,22,23,24,25].

CBPR, which necessitates an active partnership of equal commitment among various organizations throughout the entire research process, is essential in addressing health disparities such as HBV among racial/ethnic minority populations [26]. CBPR has been long implemented in other communities such as the highly noted Detroit Community-Academic Urban Research Center’s Neighborhoods Working in Partnership (NWP) [1]. In this current study, partners contributed their own expertise and worked together to improve the lives of community members living daily with HBV [1]. Similarly, for this current partnership, all members of the project team were involved in the process evaluation, which was a method used to assess the process, impact, and effectiveness of the WB-HBV Project. Findings strongly highlighted the importance for partners to clearly (1) communicate and delineate the value of the partnership and meeting structure and (2) convey the important role and contributions that community participants play and provide. It is especially critical, at the early stages, for ACG partnerships to create shared visions, norms, and equitable leadership roles for all agencies/organizations involved [27]. This helps to create buy-in to the collaboration and the partnership, and also motivation to contribute in terms of time and commitment including attending meetings. Often it is the academic partners doing the scheduling and attention needs to be given allowing the community members to do the scheduling of events and meeting with assistance from the academic partners for equitability [27, 28]. Commitment and investment of time can further be established by creating bi-directional pathways of learning; it is just as important that the ACG partnership that sets this also has a priority to elicit investments of time, attention, and attendance to the partnership [29].

The outcomes of our process evaluation comparably reflect those as described in Israel et al. (2010) [1], and were used to inform future CBPR efforts needed in DC communities to enhance capacity building and advocate policy changes. A previous process evaluation of CBPR in addressing HBV disparities highlighted that ongoing process evaluations are necessary to assess various partners’ current work, goals, and objectives and, ultimately, to achieve stronger cohesion in reaching partnership goals [30]. Specifically, there are several strategies and techniques that can increase the likelihood of partnership success [31,32,33,34,35]. As identified by Israel and colleagues [15, 23, 31], these should include (1) the facilitation of collaborative partnerships in all phases of the research, (2) the integration of knowledge and action for mutual benefit of all partners, and (3) the promotion a co-learning and empowering process that attends to social inequalities. Recognition of and responsiveness to group dynamics [31, 34, 35], including the shared development of clear and operational group goals that emphasize cooperation but reflect individual interests, contributes to the effectiveness of a diverse collaborating group, as does a climate that supports group cohesion [31,32,33,34]. In order for agreeable project aims and outcome expectations to be shared by partnership members, specific activities such as determining goals and prioritizing tasks based on theoretical frameworks [23], adopting a centralized communication network where a few members receive and share information [31], and encouraging small-group work within each organization to identify any issues and/or to regularly solicit ideas [31] are also necessary to allow for members to have a sense of group belonging and project accountability through team connectivity. Processes that facilitate equitably-distributed participation, open communication, and leadership that attends to both relationship maintenance and tasks goals are critical to the building and sustainment of effective CBPR partnerships to achieve health equity [31,32,33,34,35].

Limitations

The data presented here reflect self-reported responses belonging to a small and purposive sample of active partnership members on the WB-HBV Project Task Force. Results may have been subjected to respondent and recall bias and are not generalizable. Nonetheless, they do provide recommendations to strengthen and structure similar ACG partnerships. Moreover, results can be compared to other partnerships that used this same survey [8]. However, the mixed-method study captured additional information that may have been missed quantitatively.

Conclusions

This mixed-methods process evaluation on an ACG partnership offers critically important insights into developing strategies to enhance partnership functioning and increase the ability of this and future ACG partnerships to improve community health outcomes. The current process evaluation through a quantitative lens showed a general agreement among the three organizations across a variety of the measures adopted from Israel and colleagues, and the qualitative method allowed members to voice detailed comments regarding meeting the ultimate goal of eliminating HBV disparities in the WBMA. The results of the current process evaluation reveal strengths and weaknesses that may help to strengthen other ACG partnerships in the future.