Results of a knowledge synthesis identifying indicator domains and measures of health system integration

Introduction: Health systems integration is a cornerstone for health system reform enhancing quality of care and health outcomes. Health systems planners invest significant resources to create fully integrated health systems. However, impacts of these efforts are difficult to determine due to the lack of well-defined indicators and tools to measure progress towards integration. The goal of this study was to identify indicator domains and measurements tools to evaluate health system integration.

Methods: This international knowledge synthesis utilized a modified seven-stage scoping review approach across three sites: Rio Grande do Sul, Brazil, and Alberta and British Columbia, Canada. The knowledge synthesis built on previous work of two principle investigators (NO, ES) identifying 10 key principles for integration. A group of knowledge-users were included in discussions to validate the research questions. A panel of experts, policy and decision-makers, and providers participated in a modified Delphi process to identify relevant and important indicator domains for each principle. Three rounds of Delphi surveys were completed in English and Portuguese. Potential tools and measures for each indicator domain were identified using inclusion/exclusion criteria applied to peer-reviewed and targeted grey literature sources. Relevancy and quality were assessed using standardized tools. Data are being extracted, collated, and summarized with a final report to be completed summarizing all significant tools and measures.

Results: The Delphi process identified 16 indicator domains for nine of the ten principles. Domains included: coordinated transitions across the continuum of care; coordinated care across sectors; patient/family involvement in care planning; primary care network structures in place; team effectiveness; shared clinical pathways across the continuum of care; individualization of care pathways for patients with co-morbidities; performance measurement indicators and tools in place; clinical outcomes measured; data (e.g., administrative, performance, clinical) tracked and shared with stakeholders; shared information systems across sectors; shared patient electronic charts accessible to patients; data used for service planning; organizational goals and objectives aligned across sectors; physician integration within care teams; and goals and objectives supported by funding and human resources. No agreement was achieved by the Delphi participants on indicator domains for principle 9, governance structure. To capture tools or measures related to two or more domains an eleventh principle, overall integration, was added.

To date we have reviewed over 5500 abstracts across the 16 indicator domains. Over 115 tools have been identified for such domains as transitions in care, patient focus, intersectoral collaboration, and the use of shared clinical pathways. Where tools were not appropriate, measurement approaches and gaps in evaluation strategies have been identified. Once the literature review has been completed in its entirety we anticipate a comprehensive set of tools and measures for the indicator domains of the principles of health systems integration.

Discussion: Availability of high quality tools and measurement approaches for indicator domains varies. Some domains are more developed, such as coordinated transitions, where a large number of high quality tools have been identified. For other domains, there is a need to develop high quality tools. Tools of medium or low quality require refinement through further validity and reliability testing to assess their potential to measure domains of integration. For domains where tools were not located approaches to measure these domains require further development. For Principle 9, governance structure, the relevance of the principle to health system integration needs to be examined.

Strengths of the study include the use of the Delphi process to identify indicator domains for each of the principles by stakeholders and the thorough systematic review process conducted for each domain. Several challenges were also experienced. The Delphi process used an online survey approach; synchronous or asynchronous discussion between participants was not completed. The identification of appropriate search terms for some domains was challenging due to the variation of terms and conceptualizations used in the integration literature. Multiple searches were required to ensure the focus of the search terms were appropriate for each domain.

Conclusion: This knowledge synthesis is the first of its kind to provide a comprehensive set of tools and measures to evaluate health systems integration. The tools to measure indicator domains of integration and overall integration will provide beneficial information to policy-makers, decision-makers, and providers across a variety of health care systems internationally. Key areas for future research include: testing of tools and measures to ensure high quality tools to measure all aspects of integration; the development of further tools to measure less developed aspects of integration; and translation and validity testing of tools in different languages.