Elsevier

Public Health

Volume 129, Issue 1, January 2015, Pages 43-51
Public Health

Original Research
A quality assessment index framework for public health services: a Delphi study

https://doi.org/10.1016/j.puhe.2014.10.016Get rights and content

Highlights

  • This study was conducted by applying a two-round Delphi.

  • Consensus was reached on four first-grade, nine second-grade and 28 third-grade indices.

  • A framework of quality assessment indices for public health services was established.

  • The framework may provide guidelines for public health services in other countries.

Abstract

Objectives

This study sought consensus-based indices for quality assessment of the public health service (QAPHS) to evaluate the service quality of public health in Shenzhen and other cities in China.

Study design

A qualitative study.

Methods

A list of quality assessment indices was formed based on Donabedian theory. These indices were presented to an expert panel in a two-round Delphi study to establish a consensus view. A weight of indices was established to validate the applicability and practicability of the framework. The specialist authority coefficient and Kendall's W were also calculated based on statistical analysis.

Results

A total of 30 experts participated in the Delphi study. Consensus was reached on four first-grade indices, nine second-grade indices and 28 third-grade indices. The specialist authority coefficient (Cr) was high (between 0.88 and 0.92), while Kendall's coefficient (W) of all the indices was >0.5 with statistical significant differences (P < 0.05). This indicated correlation among panelists and had high reliability.

Conclusions

A unified and hierarchical quality assessment index framework for public health services was established. The framework should be further tested and improved in practice.

Introduction

Quality assessment in public health is the measurement of achievement of population health objectives and practices.1 Public health agencies worldwide have increasingly recognized the potential benefits of quality assessment and the need of formal assessing and improving the quality of their services.2, 3, 4, 5 Measuring quality can help monitoring progress toward public health goals and become more accountable to the populations they serve.5, 6 Public health quality indices are quantitative statements of quality assessment about the capacity, actions, or results of public health practices.1 It is important to develop a framework of quality assessment indices for public health services for the purpose of standardization and comparability.

Shenzhen is a large port city in southern China, adjacent to Hong Kong, with over 15 million inhabitants, of which about 80% being a so-called floating population (non-registered migrant population). This unique demographic situation has led to a marked diversity of public health agencies in Shenzhen, including seven agencies at city level and more than fifty agencies at district level (Fig. 1).7 For this reason, public health agencies in Shenzhen were classified differently from other cities of China. Nationwide, a set of assessment indices for public health institutions were released by China's Ministry of Health and applied generally since 2008.8

It is however, just the examination of one health department, namely ‘Center for Disease Control and Prevention (CDC)’ -- the only public health agency of many regions of China which contains all basic public health work. Those assessment indices including regional assessment indices of six categories (control and prevention of communicable diseases and chronic non-communicable disease, public health emergency disposition, health hazards monitoring and intervention, health education and health promotion, operation security) with 17 items, and institutional assessment indices of eight categories (disease control and prevention, public health emergency disposition, information management, health hazards monitoring and control, laboratory testing, health education and health promotion, technical guidance and application, and comprehensive index) with about 35 items and 100 indices in provincial, municipal and county level. However, in Shenzhen, CDC is divided into several public health institutions (Fig. 1). Therefore, the comparison to several institutions should be conducted by China's Ministry of Health rather than in one institution when designing the assessment index framework. Moreover, the assessment index framework mainly focused on the seven basic functions of CDC, and the work of functions and items of the work. There are differences between the basic functions of public health agencies, also between their work. The assessment index framework cannot be available to all public health agencies, so it ought to consider the different aspects of public health functions. Therefore, nationwide assessment indices are no longer appropriate for Shenzhen. However, several reformed assessments have been performed in Shenzhen public health agencies since 2003 separately and in a relatively independent way. Unfortunately, different public health agencies in Shenzhen have their own assessment criterions. Hence, a novel design and unified framework for quality assessment indices for public health services in Shenzhen was needed.

This study aims to establish a quality assessment index framework by applying a Delphi method, which is used for seeking consensus among experts on suitable quality assessment indices1 to provide a degree of evidence base for guidelines for quality assessment of public health services in Shenzhen and other similar areas.

Section snippets

Study design

The Delphi method is in essence an iterative multistage process that seeks to combine individuals' opinions into group consensus.9 This Delphi study was conducted over a series of two rounds in April 2010 and October 2011.

The expert panel

Purposive sampling was used to ensure that potential participants had a broad range of expertise in the fields of health administration, epidemiology and health statistics, health economics, health education, clinical medicine and social medicine. 35 experts were asked whether

Participants

Thirty-five individuals were initially contacted to be part of the Delphi expert panel. In the first round, 31 panelists returned the questionnaire, and of those 30 completed the revised questionnaire in the second round. At inclusion, there were 24 males and six females who on average had 20 years (range 5–32 years) working experience. Of these, 24 had senior positions (80%), five had secondary senior positions (17%), and one had an intermediate position (3%). All the panelists had a bachelor

Discussion

Through a two-round Delphi procedure a set of indices in three grades were established that represent a consensus based quality assessment index framework for assessing public health agencies and systems. This purpose-designed framework is based on both the aim and function of the public health system with the purpose of evaluating its performance.18, 19, 20, 21, 22 Relevant dimensions, sub-dimensions and specific indices have been defined through in-depth interviews with public health

Author statements

The authors wish to thank the contributors from Shenzhen Center for Disease Control and Prevention to this paper, specifically Richardus Jan Hendrik of Erasmus MC, University Medical Center Rotterdam for his instruction and assistance.

References (35)

  • J. Mainz

    Defining and classifying clinical indexs for quality improvement

    Int J Qual Health Care

    (2003)
  • Research group for establishment of disease control and prevention system, Research Report of establishment of disease control and prevention system[M]

    (2008 11)
  • F. Hasson et al.

    Research guidelines for the Delphi survey technique

    J Adv Nurs

    (2000)
  • A. Donabedian
    (1980)
  • C. Powell

    The Delphi technique: myths and realities

    J Adv Nurs

    (2003)
  • J. Jones et al.

    Consensus methods for medical and health services research

    BMJ

    (1995)
  • Shu-Zhong Bai

    Military medical scientific research management[M]

    (2004)
  • Cited by (0)

    View full text