Protocol for PMA-Ethiopia: A new data source for cross-sectional and longitudinal data of reproductive, maternal, and newborn health

Objectives

PMA-Ethiopia builds on the previous success of both the PMA2020 cross-sectional surveys and the PMA-MNH panel survey. With the overall goal of identifying gaps in RMNH care services, the specific objectives of the PMA-Ethiopia project are to:

A fourth objective was added in May 2020 in response to the coronavirus disease 2019 (COVID-19) pandemic.

Sample design

Cross-section. In order to assess trends since the original PMA2020 survey, the PMA-Ethiopia survey used the modern contraceptive prevalence rate (mCPR) among all women as the key indicator to derive sample size. We calculated the required number of households and women necessary to generate a 5% margin-of-error at the regional level based on the sample size calculation formula of Wilson’s method for asymmetric proportions¹⁰. Previous estimates of mCPR from the PMA2020 Ethiopia 2018 survey were used for five regions (Tigray, Amhara, Oromia, SNNP and Addis Ababa), while estimates of mCPR from the Ethiopian Demographic and Health Survey (EDHS 2016) were used for the remaining regions^4,11.

Detecting and tracking change over time in regional-level estimates is a top priority for the FMoH, thus, PMA-Ethiopia made the decision to not reduce the sample size within any region when redesigning the survey from PMA2020 to PMA-Ethiopia. Sample sizes within each region were calculated as described above; if the original sample size from the PMA2020 survey was larger than the new estimate, the original sample size was retained and the urban/rural distribution was updated to reflect the 2019 urban/rural population distribution estimated by the Ethiopian Central Statistical Agency (CSA). If the updated sample size was larger than the original PMA2020 survey, the updated estimate was used.

All EAs were drawn with probability proportional to size within the strata as noted in Table 1. For the regions of Tigray, Amhara, Oromia, and SNNP, EAs were drawn separately from urban and rural strata within the region. For the regions of Afar, Somali, Benishangul-Gumuz (BG), Gambella, and Harari, the EAs were randomly selected with probability-proportional-to-size from within the region, without urban/rural stratification. Addis is exclusively urban and thus all EAs were drawn from within the region without stratification. In total, 265 EAs were selected. Table 1 identifies the total number of EAs per region and the anticipated number of women to be interviewed in each cross-sectional survey.

Panel. EAs selected for the cross-section also serve as the EAs for the panel. The panel is designed to detect differences in health behaviors between groups “exposed” to various components of the Ethiopian health system (e.g. to detect differences in receipt of postnatal care by two days postpartum between women who were contacted by an HEW during antenatal care and those who were not). The largest sample size needed to detect a difference of 5% between groups of interest (alpha of 0.05 and power of 0.80) is 3,130, as shown in Figure 2 below. As the prevalence of the majority of RMNH indicators in Ethiopia are significantly less than 50%, sample sizes of approximately 2,500 and 3,000 are sufficient to detect differences of 5%.

Figure 2. Estimated total sample size for a two-sample proportions test to detect 5% difference.

Based on data from previous PMA2020 rounds, we assumed that approximately 10% of women age 15–49 years were pregnant or had given birth within 6 weeks. Thus, we needed to screen between 25,000 and 30,000 women. Assuming approximately 0.98 women ages 15–49 years per household and an average EA size of 175 (both estimates based on previous PMA rounds), we estimated that we would need approximately 175 EAs in the panel regions to achieve this sample size. Our final sample size of 217 EAs in the panel regions was thus deemed adequate for detecting a 5% difference between groups for the majority of RMNH indicators.

Service Delivery Point (SDP). Once EAs were identified, we obtained a list of all public and private health facilities from the local district health offices that included all health posts, health centers, and primary level hospitals in corresponding districts. While private health facilities are still relatively rare in Ethiopia, information about the RMNH services that private facilities provide is also collected. To sample private health facilities, the list of all private health facilities in each kebele, the lowest level administrative unit in Ethiopia which generally includes about 5 EAs, is reviewed. A maximum of three private SDPs within the kebele boundaries are randomly selected for interview, aligning with the previous PMA2020 protocols⁷.

All sample size calculations were conducted using Stata version 16¹².

Cross-section

The cross-sectional survey includes a short household survey, administered in each of the 35 households selected for the survey, and a longer female questionnaire, administered to every eligible female age 15–49. The female questionnaire includes questions about women’s use of contraception, experience, and perceptions of side effects related to contraceptive use, contraceptive autonomy, experience of reproductive coercion, community attitudes surrounding contraception and pregnancy, and interactions with healthcare providers related to family planning services, including whether women felt pressured by providers to use specific methods (provider bias). We also assess women’s knowledge of abortion laws and accessibility in Ethiopia.

Panel

We leverage the cohort design of PMA-Ethiopia to investigate a number of measures specific to women’s health and well-being across the continuum of care. This includes information gathered only at specific stages of pregnancy or the postpartum period (e.g. delivery, immediate postnatal care, vaccinations) and repeated measures to examine changes over time (e.g. fertility intentions). Information on individual, partner, and community factors were gathered at baseline to assess how different levels of influence affect care-seeking for RMNH.

To improve our understanding of individual and partner influences on health seeking and RMNH outcomes, questionnaires include a range of measures that we hypothesize influence women’s care seeking. These include the Conflicts and Tactics Scale (CTS-2) to measure violence during pregnancy¹⁷ and multiple measures designed to assess unintended pregnancy, including the London Measure of Unintended Pregnancy (LMUP)¹⁸. Measures from the Women’s and Girls’ Empowerment in Sexual and Reproductive Health (WGE-SRH) index are also included, including the pregnancy existence of choice sub-scale and the sexual existence of choice sub-scale, which measures constraints on the choice to become pregnant and to have sex, respectively^19,20. Similar to the cross-section, the panel study includes measures of reproductive coercion, community attitudes about pregnancy, and provider interactions for post-partum family planning services.

Service Delivery Point

The SDP survey focuses on measuring facility-based indicators of readiness to provide care across the RMNH continuum. In addition to collecting information on the availability of services and commodities, the SDP survey includes specific questions to assess facility preparedness to remove contraceptive implants, utilize health management information systems (HMIS) and referral networks with other facilities, and use Ethiopia’s integrated community case management of childhood illness (iCCM) and integrated management of newborn and child illness (IMNCI) registers to monitor the health trajectories of sick children. The SDP also collects information about aspects of the quality of health services and use of performance monitoring teams (PMTs) to regularly review facility outputs and outcomes. Finally, as SDPs are included in the sample because they serve the selected EAs, SDPs and household data can be linked via EA identifiers in both the household and SDP datasets to provide a detailed picture of the service delivery environment available to households.

ODK Programming

All of the questionnaires are programmed using ODK, an open source software for collecting and managing data. The PMA-Ethiopia team has been using this software since 2014 as part of PMA2020. Prior to data collection, all forms undergo multiple quality checks to ensure that they accurately reflect the paper questionnaire — using the same word-for-word text, question order, and skip patterns — and that the ODK forms are “bug free”.

Training

Prior to the start of the first round of data collection, PMA-Ethiopia held a series of consecutive trainings of field staff. The first was a two-week training-of-trainers (TOT) of field supervisors and regional coordinators who then served as master trainers at the resident enumerator (RE) trainings. Each RE training session was two weeks and included in-depth review of survey protocols, questionnaire content, and interview skills. In addition to classroom exercises, field staff trainings included three days of field exercises, during which participants practiced using the tools on the phone. The trainings were conducted primarily in Amharic. In addition, some small group sessions were conducted in Afan Oromo and Tigrinya. At the TOT, staff and faculty from JHSPH led the training, with support from AAU project staff, while later RE trainings were led by the AAU team and supported by JHBSPH. Additional trainings will be held throughout the course of the project.

Listing and census

In the six regions in which the cross-section and panel are implemented, a full census of the selected EAs was undertaken, listing the names, sex, and ages of all household members. Women age 15–49 within each household were consented and screened for eligibility into the panel (described further below). Upon completion of the census, it served as the sampling frame for the cross-section and 35 households within each EA were randomly selected. In the remaining regions, a household listing was conducted, identifying all households but not identifying individual household members. From this household list, 35 households were randomly selected for the cross-section.

The census and listing frames will be supplemented with new households annually prior to the selection of households for the cross-section. The census will be completely redone in 2021 prior to the baseline of the second cohort of pregnant/recently postpartum women.

Eligibility, ethics and consent

Ethical approval. PMA Ethiopia received ethical approval from Addis Ababa University, College of Health Sciences (AAU/CHS) (Ref: AAUMF 01-008) and the Johns Hopkins University Bloomberg School of Public Health (JHSPH) Institutional Review Board (FWA00000287).

Cross-section. A brief household questionnaire is administered in every household selected for the cross-section. Consent for the household questionnaire can be granted by any competent member of the household. The household questionnaire is used to collect data on age, sex, and marital status for all usual members of the household or visitors who slept in the household the night before and includes information necessary to construct a wealth index from common assets. From the household roster, all women age 15–49 years are identified and approached for consent to participate in the individual female questionnaire. The National Research Ethics Review Guidelines in Ethiopia considers women age 15–17 as able to consent for themselves when data collection covers sensitive topics, including sexual and reproductive health, so no parental consent was required²¹. Once identified as eligible, the RE describes the purpose of the study and reads the approved consent language. All consents (including panel and SDP, described further below) are provided as oral consent per guidance from the National Research Ethics Review Guidelines. Based on this guidance and from the IRB on record, written consent is not required in areas of low literacy or when data collection does not include invasive procedures (e.g. biospecimen collection). Data collection procedures were approved by the AAU and JHBSPH Institutional Review Boards.

Panel. All women age 15–49 years who are identified in the census are consented and screened for eligibility. Women are eligible for the panel if they are regular members of the household, including women staying at their parental home for the delivery and postpartum period, and self-identified as currently pregnant or less than six weeks postpartum. At screening, the RE explains the purpose of the survey and cadence of interviews. If the woman consented, a household questionnaire, followed by a baseline questionnaire are conducted, resulting in baseline data collected across a range of gestational ages, from less than one month pregnant to six weeks postpartum.

For women who are pregnant at baseline, their approximate gestational age is used to estimate date of delivery. REs contact the respondent at frequent intervals to monitor whether the delivery has occurred based on the expected date of delivery and conduct the first follow-up interview at approximately six weeks postpartum. In the event that the respondent does not have a phone or access to a phone, REs relies on the local Health Development Army (HDA), a group of community-based volunteers who monitor pregnant women in the community and assist Health Extension Workers (HEWs). In the case of pre-term/early deliveries, families are encouraged to inform the REs and HDAs. Once the 6-week interview is complete, the 6-month and 1-year interviews are scheduled. A week before each scheduled interview, the data management team sends an automated short text message in Amharic to each RE to remind them of the scheduled follow-up date. Data are collected within two to three weeks of the targeted interview date for each of the follow-up interviews. Women are initially approached for consent at the baseline interview, when REs describe the purpose of the study and read the approved consent language. At each follow-up interviews, women are asked if they have any questions and if they still agree to participate in the study, but the full consent language is not read again.

SDP. All facilities identified as serving the EA through the sampling process described above are eligible for participation. A letter of support from regional and zonal health authorities is provided to all interviewers and consent is granted by the head of the facility or designated authority. In smaller facilities (e.g. health post, pharmacies, and drug shops), interviewees are generally the in-charge or owner of the facility. In larger facilities (e.g. hospitals, health centers), multiple respondents, generally the in-charge of each unit, provide information.

Data management

Data are downloaded from the ODK Aggregate server daily and cleaned by the data management team using Stata version 16¹². QGIS version 3.14 is used to review the distance between each household and the center of the EA to identify interviews that may have occurred outside of the study area²².

When an error is suspected, the team communicates with the interviewer to confirm whether she entered the correct response, and in some cases to ask her to provide a correction. The team manually corrects these errors using Stata version 16 during data collection so that few errors remain by the end of field work.

Weighting

Sample weights are constructed based on the selection probabilities of the EAs provided by the CSA. After data collection for each survey is complete, two weights (household and female) are created in the cross-section and two weights in the panel (household and female) to adjust for selection probability and non-response.

Cross-section. The household weight is the inverse of the probability of selecting the EA multiplied by the probability of selecting the household within the EA. The weights are further adjusted to account for the non-response rate at the household level within the EA. As all females ages 15–49 years within selected households are surveyed, there is no further selection probability for the female. Instead to arrive at the final female weight, the household weight is adjusted for non-response of the female questionnaire among all eligible females within the EA. Female weights are created for de facto females only (i.e. women who slept in the household the night before the survey was administered). Application of the PMA-Ethiopia household and female survey weights for the cross-section result in a sample that is nationally representative of households and females age 15–49, respectively.

Panel. As all households are included in the census, there is no additional selection probability of households; thus, the household weight is the inverse of the EA selection probability and the response rate to the census within the EA. Female weights for women in the panel are adjusted for non-response within the EA, and follow-up surveys will adjust for loss to follow-up from the baseline panel survey sample. Application of the PMA-Ethiopia household and female survey weights for the panel survey will result in a sample that is representative of all households with a pregnant or recently postpartum women and all pregnant or recently postpartum women age 15–49 residing in the six regions included in the PMA-Ethiopia panel, respectively.

Service Delivery Point. No weights are applied in the SDP survey as the sample is not selected probabilistically from a known sampling frame.

Data availability

Anonymized microdatasets from PMA-Ethiopia will be made publicly available within approximately six months of the close of each phase of data collection (e.g. data from 6-week interviews will be made available six months after the final 6-week interview is completed). Datasets will be available for download via the www.pmadata.org website and will be available in csv, xls, and dta format. Data cleaning and release follow the same principles of the PMA2020 survey⁷.

To contribute to ongoing efforts to improve coverage estimates⁶, one GPS point per cluster will be made available, which will be randomly displaced following the same procedures used in other PMA surveys²³. All SDP surveys include a variable indicating which PMA-Ethiopia EAs fall within its catchment area, allowing for linkages between health facilities and households.

Statistical analysis

Preliminary data analysis will focus on computing frequencies, proportions, means or medians. Characteristics of survey participants will be described by computing frequencies and percentages for categorical variables. The initial analytic approach will be largely exploratory for data quality checking, including non-response/missingness and the extent of “don’t know” responses.

For both the cross-sectional and the longitudinal anlayses, descriptive analyses will examine proportions, tabulations, and summary measures (mean, median, range, etc.) for summarizing binary, categorical, and continuous variables, respectively. Subgroup analyses will be performed when it is feasible (e.g., adequate sample sizes in the contingency table cells). The descriptive analysis is particularly relevant for the coverage estimates of specific health services which are of primary importance for the FMoH (i.e. delivery in a health facility, receipt of postnatal care, receipt of specific vaccines, etc.).

Time to event analyses (life table and hazard models) will be used to assess time to uptake of postpartum family planning services, median time of exclusive breastfeeding and contraceptive discontinuation and switching within one year postpartum. Use of health services and receipt of specific components of care (i.e. receipt of postpartum family planning counseling, receipt of exclusive breastfeeding counseling) will be the primary independent variables to assess differences in utilization/duration, adjusted for confounding individual and partner characteristics.

We will conduct trend analysis to assess changes in vaccination completion rate per the vaccination schedule, modern contraceptive prevalence rate, contraceptive unmet need, and other high-impact intervention indicators (i.e. received treatment for maternal complications, immediate newborn care, etc.) over the four years of the study.

To assess the reliability and validity of the reproductive empowerment scale items, analysis of item performance, factor analysis, and association of the empowerment measure with specific outcomes will be evaluated. More detail is provided below.

Analysis of item performance and factor analysis: For the questions on reproductive empowerment, item-scale correlations, means, and variances will be computed. Internal consistency will be determined using Cronbach’s alpha coefficient²⁴. Principal Component Analysis (PCA), Parallel Analysis (PA), and Exploratory Factor Analysis (EFA) will be conducted based on polychoric correlation matrices. Factor analysis will also guide the selection of a more parsimonious set of items loading on the identified factors (based on 0.40 factor loading criteria)²⁵. Cronbach’s alpha values of 0.70 or higher will help determine the internal consistency of the subscale and scale items.

All analyses will be performed in Stata or R, per analyst preference. All statistical tests will be done at a significant level of p≤0.05. The data will be presented in text narratives, tables and graphical presentations.

Dissemination of information

Key results from the cross-section, baseline, and SDP surveys will be presented in briefs and in PowerPoint presentations and made available at https://www.pmadata.org/countries/ethiopia. Results will include an overview of service coverage and comprehensiveness, family planning indicators, and availability of essential RMNH and FP commodities. Results related to delivery, postnatal care, nutrition, and vaccine coverage will be made available as follow-up interviews are conducted.

Findings from inferential analyses will be published in open-access, peer-reviewed journals. Findings of significant importance to the FMoH and PAB will be discussed at PAB meetings to inform policy and program response and identify future research trajectories.

Study status

Data collection is ongoing, with the study scheduled to end in 2023.

Underlying data

No data are associated with this article.

Extended data

All questionnaires are publicly available on the www.pmadata.org website.

The 2019 Service Delivery Point Questionnaire is available at https://doi.org/10.34976/kvvr-t814¹³

The Panel Questionnaire is available at https://doi.org/10.34976/h75w-8084¹⁴

The 2019 Cross-Sectional Household and combined Female Questionnaire is available at https://doi.org/10.34976/6hen-vd80¹⁵

To gain access to datasets when they become available you will need to create a profile and apply for access via this link - https://www.pmadata.org/data/available-datasets/request-access-datasets. When data collection is finished, all datasets will be made indexed and made available through the www.pmadata.org website.