INTRODUCTION

In response to rising healthcare costs, many healthcare systems have started implementing intensive primary care programs for high-need patients with medical, social, and behavioral complexity.1,2,3,4,5 These programs typically include proactive care coordination and case management to address medical issues, care fragmentation, care transitions, and other healthcare navigation challenges.6

Evidence of intensive primary care’s impact on healthcare utilization patterns is mixed.4, 5, 7, 8 A recent systematic review of 18 studies by Edwards et al. found that most intensive primary care programs had no impact on mortality or emergency department use, and the effectiveness in reducing hospitalizations varied.8 Some of the most promising findings around acute care use have emerged from programs for older adults that have reported reductions in hospitalizations and/or length of stay.9,10,11

The effects of intensive primary care on quality of care and quality of life are more promising. In a 2009 Institute of Medicine review, Boult et al. found that comprehensive care for older adults frequently had positive effects for quality of life.12 For example, the GRACE model was associated with improvements in quality indicators for general healthcare and geriatric conditions, and improvements in vitality, social function, and mental health.13 In a pilot Veterans Affairs (VA) intensive primary care program, enrollment was associated with significantly more visits with an assigned primary care physician, higher rates of advance directive completion, and higher rates of hospice referral near end of life.14, 15

Fewer studies have examined the impact of intensive primary care on patient experience. A recent evaluation of Comprehensive Primary Care among fee-for-service Medicare beneficiaries found that there was no appreciable improvement in beneficiary experience compared to individuals in matched practices.16 A limited number of randomized controlled trials of intensive primary care suggest potential benefits for patient satisfaction with specific elements of care, such as transitions17 and access to telephone advice.18 However, there have been few rigorously designed trials that have included a comprehensive assessment of the experiences of patients receiving intensive management compared to those receiving usual primary care.

To address the gap in evidence about patient experiences with intensive primary care, we conducted a survey of patients in a multi-site randomized quality improvement trial of intensive primary care in the VA healthcare system. Previously, we found that access to the VA program increased use of primary care, social work, and mental health services, with no increase in total costs.19 In this study, we hypothesized that the program positively influenced patient perceptions of their care coordination, and related experience measures such as satisfaction and perceived access.

METHODS

This study examines the results of a survey conducted as part of an evaluation of a five-site VA intensive primary care demonstration project. One of the largest integrated healthcare systems in the USA, the VA provides care to more than eight million eligible Veterans. Since 2010, VA primary care has been delivered by Patient Aligned Care Teams (PACTs), which are structured based on a patient-centered medical home model.20 PACT teamlets include a primary care provider, nurse, clinical associate, and administrative associate, and are supported by social work, pharmacy, and behavioral health services.

While PACT implementation improved patient satisfaction and reduced acute care utilization,21 VA leadership recognized that additional resources were required to meet the needs of Veterans with particularly complex and high-risk profiles. In 2014, VA’s Office of Primary Care implemented the PACT-Intensive Management (PIM) Demonstration Program to augment PACT services for Veterans at high risk for hospitalization (i.e., individuals whose risk of 90-day hospitalization was ≥ 90th percentile based on the VA’s Care Assessment Need (CAN) score)22 and who had experienced a hospitalization or emergency department (ED) visit in the previous 6 months.

The PIM Demonstration Program has been described previously.19, 23 Building on effective intensive outpatient interventions such as GRACE,13 a VA Intensive Management PACT (ImPACT) pilot,14, 15, 24, 25 and lessons learned from other early programs,4, 5, 7 the five PIM programs incorporated features such as regular interdisciplinary team meetings, medication management, home visits, mental health/substance use assessment and support, health coaching, and intensive social work case management. In addition, the teams engaged in a number of specific activities to coordinate care within VA and across VA and other settings (Table 1).

Table 1 Examples of Care Coordination Activities Among Five PIM Sites
Full size table

Design of PIM Evaluation

The PIM Demonstration Program and associated utilization and cost evaluations were designed to support VA leadership priorities and were therefore designated as quality improvement (QI) by VA’s Office of Research and Development and Office of Research Oversight (#NCT03100526).19 Because the QI evaluation was restricted to analyses of data collected for PIM or PACT clinical care,23 we obtained approval for an additional patient survey from Stanford University’s Institutional Review Board and VA Palo Alto’s Research and Development Committee.

Survey Sample

The survey sample was drawn from a list of high-risk Veterans who were randomized to PIM or usual PACT care between August 2014 and December 2015 at the five VA facilities with PIM Demonstration Programs. The survey was conducted from June 3, 2016, to December 16, 2016. Veterans were eligible to participate in the survey if they had a valid mailing address or telephone number and could answer survey questions. Veterans unable to respond to the printed or telephone survey had the option of designating a caregiver to complete the survey on their behalf. Of the 2566 potential participants, 38 were not eligible (i.e., deceased or did not have a valid mailing address or telephone number). Of the remaining sample, 1527 (60%) completed the survey; 737 individuals could not be contacted by mail or telephone; and 264 individuals opted out of the mailed survey or refused telephone follow-up (Appendix 1).

Survey Procedures

All eligible Veterans were mailed a survey packet containing the survey, a cover letter, an information sheet describing the study’s risks and benefits, a one-time $2 pre-incentive, and an opt-out form. Non-respondents received a second packet, a reminder postcard, then a third packet, and finally a postcard informing the recipient that he or she would be contacted by telephone. A completed survey indicated informed consent to participate. Respondents were mailed a $10 check in appreciation of their time.

Survey Measures

Survey measures were derived from widely used patient surveys,26,27,28 and validated measures of care coordination from the literature.29, 30 Patient experiences with care coordination were assessed using questions from VA’s Survey of Healthcare Experiences of Patients (SHEP),26 adapted from the Consumer Assessment of Healthcare Providers and Systems (CAHPS).27 Patients were asked whether in the last 6 months someone had talked to them about their specific goals for their health and things that make it hard for them to take care of their health, whether someone had asked them about their medications, and whether they had received reminders about tests, treatment, or appointments between clinic visits. Responses were analyzed using standard SHEP procedures (i.e., percentage that responded “yes”). Patients were also asked whether their VA primary care providers seemed informed about the care they got from specialists (analyzed using top-box scoring based on percentage that responded “always”) and whether they had a VA healthcare provider who helped “coordinate care from different doctors and services” (analyzed using top-box scoring based on percentage that responded, “agree strongly”).

We also examined two validated measures that broadly consider patient experiences with their care and care coordination. The first was the Patient Assessment of Chronic Illness Care (PACIC),29 a six-item measure (analyzed as a composite score) that asks about care coordination activities related to chronic illnesses, such as contact after a visit and discussion about how specialists are contributing to treatment. The second was the Health Care Hassles Scale,30 which queries patients about diverse challenges such as insufficient information about treatment options, difficulties refilling medications, and perceptions that concerns were overlooked (responses range from 0 (not a problem) to 4 (very big problem), and the scale is analyzed as a composite score that ranges from 0 to 60).

Access and relationships with providers were assessed by asking whether patients agreed with the statements “I got the service I needed” and “it was easy to get what I needed”26 and with the adapted questions, “I have a VA healthcare provider who I can contact when I have questions about my care,” “I have a VA healthcare provider who I trust,” and “I have a VA healthcare provider who respects me.” Responses were analyzed using standard SHEP procedure: the percentage who responded “agree strongly.”

Satisfaction with VA care (including overall care, and primary care, mental healthcare, and social services) was assessed using questions adapted from the 2013 Customer Satisfaction Index.28 Respondents rated their experiences on a scale of 1 (very dissatisfied) to 10 (very satisfied). We examined the proportion that rated their satisfaction 10 out of 10, and the mean satisfaction ratings after converting ratings to a 0–100 scale.

Finally, because only a subset of randomized patients participated in PIM, survey respondents were queried whether they were currently or previously enrolled in PIM. Those who endorsed enrollment were asked about their satisfaction with the program, and whether the PIM team was responsive to their health concerns, helped them get VA care and services that they needed, improved their overall care with VA, and helped them meet their goals.

Covariates assessed through the survey included social support (evaluated using the Medical Outcomes Survey questions about tangible social support),31 loneliness and isolation (measured using the R-UCLA Loneliness Scale),32 health literacy (measured using Chew et al.’s single item),33 functional status (assessed using the National Institute of Health’s PROMIS Short Form Physical Function 4a measure),34 education, and income. Other covariates obtained from VA’s national Corporate Data Warehouse included age, sex, race/ethnicity, marital status, and clinical complexity (as measured by the average risk for 90-day hospitalization during the 6 months prior to randomization, based on the VA’s CAN score).22

Statistical Analyses

We used chi-square and t tests to compare sociodemographic and clinical characteristics among all survey respondents and non-respondents. For patients randomized to PIM vs. PACT, we compared administrative data-based sociodemographic and clinical characteristics, and survey-based measures of health literacy, social support, education, and income. We describe responses for each outcome measure for three groups: patients randomized to PIM, patients randomized to PACT, and the subset of patients randomized to PIM who reported that they were currently or previously enrolled in the program.

In primary intention-to-treat analyses, we conducted logistic regression models to examine whether randomization to PIM was associated with differences in patient experiences. Separate models were conducted for each outcome within the domains of care coordination, provider relationships, access, and satisfaction. To account for potential respondent bias, we used logistic regression to calculate the probability of response for PIM and PACT patients, then used inverse probability weighting to account for low response in each group. Models were run using PROC SURVEY LOGISTIC in SAS. In sensitivity analyses, we included site fixed effects and adjusted models for demographic and clinical covariates that differed between PIM and PACT patients with a P value < 0.2. All covariates were missing at rates < 5%. In order to determine whether PIM effects varied by certain patient characteristics, we conducted secondary analyses to assess the first-order interaction term for model covariates that were significantly associated with study outcomes. All tests were two-sided and conducted at the 0.05 level of significance, with an odds ratio of 1.5 constituting a small effect based on Cohen’s conventions for effect size.35, 36

RESULTS

Survey respondents were older than non-respondents (mean (SD) age of 64.4 (11.5) vs. 61.8 (13.5); P < 0.001) and were more likely to be married (34.1% vs. 28.7%; P = 0.005), but were similar in terms of sex, race/ethnicity, and clinical complexity (Appendix 2). There were no significant differences between PIM (N = 759) and PACT (N = 768) survey respondents across any sociodemographic or clinical characteristics (Table 2).

Table 2 Sociodemographic and Clinical Characteristics of PIM and PACT Survey Respondents
Full size table

Patient Perceptions of Care Coordination

Patients randomized to PIM were more likely than those in PACT to report that someone talked to them about their health goals (73.1% vs. 68.4%; AOR (95% CI) = 1.26 (1.00, 1.59)) and barriers to taking care of their health (60.4% vs. 54.8%; AOR (95% CI) = 1.26 (1.02, 1.56)) (Fig. 1). PIM patients also had higher mean (SD) scores for the PACIC chronic illness care scale (2.91 (1.31) in PIM vs. 2.75 (1.25) in PACT; P = 0.022) but not for the mean (SD) Health Care Hassles scale (9.87 (11.83) in PIM vs. 10.17 (11.24) in PACT; P = 0.61) (see Appendix 3 for results for individual items). Other differences in coordination measures were not statistically significant (Fig. 1, Table 3).

Figure 1
figure 1

Experiences of patients randomly allocated to PIM (N = 759) vs. PACT (N = 768). Superscript letter “a” indicates percentage of patients who responded “yes” when asked about receipt of services in the past 6 months. Superscript letter “b” indicates percentage of patients who responded that they “always” experience this element in their care. Superscript letter “c” indicates percentage of patients who responded that they “agree strongly” that they received or experienced the specified care.

Full size image
Table 3 Effect of Intensive Primary Care on the Experiences of High-Need Patients in PIM (N = 759) vs. PACT (N = 768)
Full size table

Provider Relationships and Access

Patients randomized to PIM were more likely than patients in PACT to strongly agree that they have a VA healthcare provider whom they trust (60.5% vs. 53.1%; AOR (95% CI) = 1.35 (1.10, 1.66)). There were no other statistically significant differences in perceptions of relationships or access, including receipt of needed services, ease of getting needed services, and having an accessible VA provider when questions about care arise (Fig. 1, Table 3).

Satisfaction

PIM patients were more likely than PACT patients to report 10 out of 10 on satisfaction with primary care (36.5% vs. 31.7%; AOR (95% CI) = 1.25 (1.05, 1.47)) (Table 3). There were no significant differences in PIM vs. PACT patients’ mean satisfaction ratings of their overall VA care, or component primary care, mental healthcare, or social services (Appendix 3).

PIM Program Participant Assessment of PIM Services

Among the 759 patients who were randomized to PIM, 732 responded to a question about enrollment, and 281 (38%) reported that they were currently or previously enrolled in the program (Appendix 1). Among the self-reported enrolled patients, 46% (124/269) reported 10 out of 10 on satisfaction with PIM, and a majority reported that the PIM team was responsive to their health concerns (80%, 219/273), helped them get VA care and services that they need (82%, 224/273), improved their overall care with VA (72%, 195/270), and helped them meet their health goals (70%, 190/270).

Sensitivity Analyses and Interaction Terms

Sensitivity analyses with site fixed effects and adjustment for sociodemographic and clinical factors that differed between PIM and PACT patients with a P value < 0.2 (i.e., age and functional status) yielded similar results (Appendix 4). Secondary analyses assessed the first-order interaction term for tangible social support and greater functional status (which were associated with positive patient experience outcomes in most models), but the interaction terms for each of these covariates with the intervention were nonsignificant for the majority of outcomes (results not shown). Finally, we examined the responses of the 38% of PIM patients who reported current or previous PIM enrollment. Appendix 5 illustrates that the responses of these patients were slightly higher than the responses for the overall population randomized to PIM for 9 out of 11 measures of care coordination, relationships with providers, and access.

DISCUSSION

This study represents one of the first comprehensive assessments comparing the care experiences of patients in intensive primary care with those of patients receiving usual care in a patient-centered medical home. Previously, we found that the PIM program offered high-need Veterans more primary care, social work, and mental health services, with no increase in total costs.19 Survey findings suggest that the program may have influenced some patients’ experiences with patient-centered care and chronic illness care, and increased the number of patients who reported having a trusted provider, but did not influence satisfaction, perceived access, or most measures of care coordination.

Taken as a whole, these findings suggest that the program had a modest effect in domains that were related to relationship-building (i.e., having a trusted provider) and patient-centeredness (i.e., assessing patient’s health goals), as well as perceived chronic illness care. Trusting relationships have been described as the basis for engaging patients in case management and primary care, particularly when attempting to engage patients with complex medical, social, and behavioral needs.37 Furthermore, relationships are at the core of primary care, so this finding suggests that augmenting a medical home with an intensive management program may help fulfill the promise or primary care. In fact, analyses of satisfaction suggest that the program improved patients’ experiences with primary care, but not with other services. Improving primary care processes could potentially have positive long-term consequences, including changes in health behaviors and clinical outcomes.38

One explanation for why the program did not have a greater effect on care coordination experiences is that the VA’s medical home already addresses basic coordination needs such as medication review and providing reminders between visits, so the greatest opportunity for improvement is in offering patients more time with an attentive team that can conduct in-depth assessments of their goals, priorities, barriers, and challenges.

It is notable that only 38% of patients who were randomized to PIM reported that they were enrolled in the program. Previous evaluations have documented that over half of those randomized to PIM were identified as not needing the intervention, were offered the intervention but declined, or received fewer than three encounters.19 Other patients may not have distinguished between PIM and their usual PACT care, which already included a number of ancillary services. Among the patients who reported receiving PIM services, nearly half reported 10 out of 10 on satisfaction with the program, and a majority felt that the program was responsive to their concerns and helped them meet their health goals and obtain needed services. While these patients represent a minority of the intention-to-treat population, their responses appear to drive the differences between outcomes for the randomized populations (Appendix 5).

Our study was limited to five VA primary care sites that chose to participate in an intensive primary care demonstration program. VA facility-level data for four SHEP questions in the survey indicate that three of the five sites had ratings that were at least 5% higher than the national average for three of the four questions. As a result, findings may not be generalizable to VA (or non-VA) sites with poor patient experience ratings at baseline, where a resource-intensive program could potentially have a greater impact. In addition, there were 17 tests included in our primary analyses, each of which was examined in two sensitivity analyses. Overall, the findings for trust and patient-centered care outcomes were robust even in the most conservative models; however, results that were of borderline significance in the main analyses, such as patient experiences with care coordination logistics, should be interpreted with caution.

To our knowledge, this is the first multi-site randomized trial to investigate the effects of intensive primary care on such a broad array of patient experience outcomes. Our findings suggest that augmenting a medical home with intensive primary care can increase the number of high-need patients who report having a provider whom they trust, and may improve patient-centered care coordination and perceived chronic illness care. While the overall effects on population-level patient experiences were modest, these domains are aligned with current VA priorities and may have a long-term impact on clinical outcomes in this vulnerable patient population.